DEV Community: Robin-Manuel Thiel

With the fall of Stack Overflow, AI Coding Assistants like GitHub Copilot will have a data problem!

Robin-Manuel Thiel — Sun, 01 Sep 2024 12:42:47 +0000

Stack Overflow shares website data with some of its most active members (how cool is that?), which is currently showing a dramatic decrease in traffic, which ML engineer Ayhan Fuat Çelik named "The Fall of Stack Overflow". As one of the biggest data sources of LLM trainings (especially those focussing on assisting with coding), Stack Overflow plays a critical role in the success of AI Coding Assistants like GitHub Copilot. Even though Gergely Orosz pointed out, that the reports of Stack Overflow's downfall might be exaggerated, the number of questions being asked on the platform is indeed decreasing.

With fewer questions about current programming problems being asked on the public internet, the training data for the coding assistants of tomorrow gets reduced. Ironically, the coding assistants of today are one of the main reasons for the fall of Stack Overflow and why people ask their questions in private to an AI.

Why could this become problematic?

AI Coding Assistants are based on Generative AI Models, which can only be as good as their training data. With less public knowledge on coding, how do we train models to solve the coding challenges of the programming languages, frameworks, and tools of the future?

Developers don't have fewer questions, they just ask them to AI Assistants instead. Even if these AI Assistants would publish anonymized questions and answers to a public data set, it would not be good advice to train new AI Models on them, as training AI on AI-generated content risks poisoning the training data for future AI models. This can lead to a phenomenon called "model collapse," where errors accumulate over generations, resulting in nonsensical outputs. Experts warn that as AI content becomes more prevalent, it could bias models and degrade their quality.

Will AI Assistants have a Netflix Problem?

With human-created data-sets seeing a dramatic increase in value for training Large Language Models like Open AI's GPT family, we are already seeing owners of these training sets like Reddit guarding them and blocking automatic access.

In my opinion, the worst outcome of all this from a user experience perspective would be, if companies like Reddit or Stack Overflow came up with their own AI Assistants, which would then have exclusive access to their data. This is what I call the Netflix problem, referring to a time when Netflix had all existing shows and movies in their portfolio, and now we ended up with Amazon Prime Video, Disney+, Paramount and many more. The same happened to car sharing and E-Scooters.

People don't want to ask 5 different AI coding Assistants for help, hoping that one of them was trained on data that could lead to an answer. They want one. There could be different ones on the market, but once they decide on one, they want to use it for everything.

Or maybe it is not a big problem at all?

From my own experience, AI Coding Assistants work fantastic for narrowly scoped and straightforward questions. Whenever an issue becomes complex or might happen across multiple layers like programming language, framework, database driver, database type and so on, they still struggle, and I turn to Stack Overflow.

We should probably take a look at which kind of questions are being asked less on Stack Overflow. If had to guess, I would say it's the straightforward and less complex ones. Maybe these could also be answered by AI models by taking the official and public documentation into their training sets.

I'm sure, all this also applies to other industries that use AI Assistants and Copilots already. I am very curious to see, how this affects the quality of future AI applications.

What we learned after spending 5,000 EUR on developer marketing

Robin-Manuel Thiel — Tue, 27 Aug 2024 13:18:23 +0000

For Space Blocks, a Developer Platform for Permission Management, we decided to spread the work through online marketing in various channels like Social Media, Google Ads, Newsletters, and YouTube Video Ads. Here is our wrap-up.

The campaigns

We are targeting Software Developers with our product, and our main goal for the campaign was raising awareness of our existence. We wanted to let developers know that we offer an easy-to-use solution for all kinds of permission related features. We gave ourselves a budget of €5,000 and a time-span of one month to evaluate what I am now sharing here.

Google Ads 👍

We didn't hear many success stories about Google Ads so far, and so we have been careful with our spendings there, but still wanted to give it a try. Navigating through the Google Ads Console is a pain and although you can probably configure everything, it is very overwhelming and contains a lot of new ad-specific vocabulary, which made it challenging to understand for beginners like us. Google lets you target specific locations (countries) and keyword "themes", from which they derive the actual keywords they are showing your ad on. You can edit those keywords later throughout the running campaign. If your ad will be shown, when a user searches for one of these keywords, depends on how much money you are willing to spend.

We used the following configuration for the campaign.

Countries: Latvia, Andorra, Liechtenstein, Armenia, Lithuania, Austria, Luxembourg, Azerbaijan, Malta, Moldova, Belgium, Monaco, Bulgaria, Netherlands, Croatia, Norway, Cyprus, Poland, Czech Republic, Portugal, Denmark, Romania, Estonia, Finland, San Marino, France, Slovakia, Georgia, Slovenia, Germany, Spain, Greece, Hungary, Sweden, Iceland, Switzerland, Ireland, Turkey, Italy, United Kingdom, United Arab Emirates, United States
Keyword themes: app development, Access Control, software development, permissions, RBAC, access rights, permission checks, permissions for apps, permissions for developers
Max daily budget: $20

With these values, we were able to achieve 360,000 impressions and 10,900 clicks over a total cost of €448. Not a crazy good result, but spending less than $500 to make 360,000 people aware of our product and have more than 10,000 people actually visiting the website is not too bad.

Non-skippable YouTube Video Ads 👎

As at Space Blocks we are all big fans of watching YouTube videos on Technology and Programming content, we thought running video ads before those video was a good idea. When you create a Video Spot that is less than 15s, YouTube even lets you create it as a non-skippable(!) ad out of it, which sounded very appealing to us.

YouTube Ads are scheduled through the same Google Ads Console as Google Ads and basically work the same in terms of targeting.

Countries: Latvia, Andorra, Liechtenstein, Armenia, Lithuania, Austria, Luxembourg, Azerbaijan, Malta, Moldova, Belgium, Monaco, Bulgaria, Netherlands, Croatia, Norway, Cyprus, Poland, Czech Republic, Portugal, Denmark, Romania, Estonia, Finland, San Marino, France, Slovakia, Georgia, Slovenia, Germany, Spain, Greece, Hungary, Sweden, Iceland, Switzerland, Ireland, Turkey, Italy, United Kingdom, United Arab Emirates, United States
Audiences: Technology Industry, Business Professionals, Technology, Cloud Services Power Users, Business Technology, Hosted Data & Cloud, Enterprise Software, Web Services, Web Design & Development, Business Creation, Recently started a Business, Starting a Business soon, All Users of Space Blocks Website

The results were disappointing. Although spending much more money on YouTube than on Google Ads, only one third of impressions (105,000) were generated. With not even 500 clicks for more than €1,200, YouTube Ads had the worst performance of the entire campaign, in contrast to the highest production cost and time of the video spot. (It turns out, that fitting content into 15s is much harder than expected, as every single frame counts).

Reddit Ads 👍

As Reddit is popular among developers, we also wanted to run ads there. The experience of creating ads on Reddit, as well as their performance, was surprisingly pleasant in contrast to Google.

Reddit lets you target precisely in which Subreddits you want to display your ad and offers to extend this selection dynamically using their algorithm, which we accepted.

Countries: Latvia, Andorra, Liechtenstein, Armenia, Lithuania, Austria, Luxembourg, Azerbaijan, Malta, Moldova, Belgium, Monaco, Bulgaria, Netherlands, Croatia, Norway, Cyprus, Poland, Czech Republic, Portugal, Denmark, Romania, Estonia, Finland, San Marino, France, Slovakia, Georgia, Slovenia, Germany, Spain, Greece, Hungary, Sweden, Iceland, Switzerland, Ireland, Turkey, Italy, United Kingdom, United Arab Emirates, United States
Subreddits: r/javascript, r/aws, r/developer, r/googlecloud, r/node, r/dotnet, r/reactjs, r/SaaS, r/devops, r/softwaredevelopment, r/ChatGPTCoding, r/cloudcomputing, r/programming, r/webdev, r/AZURE, r/csharp, r/LangChain, r/Cloud, r/micro_saas, r/AppDevelopers, r/AskProgramming, r/WebDeveloper (dynamic extension activated)
Cost-cap: €2.00 per click

With a budget of €700, we were able to reach nearly 1.5 Million developers, from which approx. 3,000 clicked the ad and visited our website. This is many more impressions than on Google, but also fewer clicks for a comparable budget. For impressions, I would always choose Reddit, although the idea, what many of them might have been bots scraping Reddit leaves a bitter taste and might explain, why we were seeing fewer clicks. Bots usually don't click on ads.

DEV.to Ads 👎

As regular readers of DEV.to, we also wanted to advertise there. We chose the smallest available campaign size, which runs for 7 days for $1,000.

*This pricing tier does not allow to segment the target audience *(the larger ones do) but as on DEV.to, there are only developers anyway, this didn't bother us too much.

Over the 7 days that the campaign was running, we were able to generate 101,075 impressions and only 112 clicks. So either our banner ad was not very appealing (maybe we used too many emojis?) or DEV.to users just don't click on Banner Ads.

Hacker Newsletter Sponsoring 👎

One last channel, we tried was newsletter sponsoring. I don't have the feeling, developers are overly into E-Mail newsletters, but we wanted to give it a try. The Hacker Newsletter has 60,000 subscribers and collects trending discussions on Y Combinator's popular Hacker News website.

For $200, you can buy a "classified" (promoted) link for one issue of the newsletter. We gave it a try and did not register a single page visit from there.

A word on costs and offers

If you sum up all the spendings in this article, you will come to the conclusion, that in total we spent more than €5,000. In reality though, both, Google and Reddit had special offerings at the time when we were running the campaigns, that we used.

Google offered an additional credit of €800, when spending more than €1,200 on their ad platform. Reddit had a similar deal and offered $200 credits, after spending at least $100. Look out for this kind of deals, when running your own campaign.

What about Ad Blockers?

Nearly all developers are very tech-savvy users and, I would guess, the vast majority of them use ad blockers (I do so myself). So it's a valid question, if banner ads are the right medium to reach them in the first place. I don't have a finite answer on that.

Ad Blockers are the main reason why we chose YouTube Video Ads, as YouTube makes it increasingly hard to use Ad Blockers since they try to sell their Premium plan. The same goes for other media formats like Newsletters and Podcasts. Maybe a Banner Ad is not the right format anymore and our brains are so used to filtering them out? 🤔

Hint: If you plan to run ads for any kind yourself, make sure to disable your own ad-blocker. Otherwise, most ad platforms won't work properly. Don't ask me, how long it took me to find out...

Measure your success

If you are really just after brand awareness, tracking impressions and clicks might be enough, but in most cases, you will want to know, how users interact with your website or product after clicking on an ad (e.g. pricing page visited, signed-up or booked-demo). For this, nearly all ad platforms let you define campaign goals in the form of events.

You should absolutely track these events by embedding the according reporting scripts, which are often called pixels (Facebook Pixel, Reddit Pixel) into your website, so that the reporting tools of your ad platform can track them. It is advisable to also track the source of the traffic (e.g. Newsletter X, banner ad on website Y) with tools like the Google Analytics URL Builder, in case an ad platform does not support event tracking. This gives each ad network a unique campaign link, and you can also validate their reports on your end.

Summary

In total, we reached approx. 2,000,000 users (impressions) and could show them the existence of our product, from which we could convince 14,000 users to click on one of our ads and visit our website. With a budget of only €5,000, I call this a mild success, but not a huge one.

It is worth to mention that Google Analytics tracked much fewer sessions than the ad networks reported clicks, which led us to the conclusion, that some clickers closed the website immediately. It is advisable to track the source of the traffic (e.g. with the Google Analytics URL Builder) and give each ad network a unique campaign link.

In summary, banner and video ads targeted towards developers seem to work much better for awareness than for clicks - meaning if you want to let developers know, that your product exists, it might be a useful tool, but if your goal is engagement, I wouldn't count our campaigns as a success.

Understanding the Difference Between Authentication and Authorization

Robin-Manuel Thiel — Thu, 09 May 2024 07:29:00 +0000

In the realm of software development, particularly in the context of security, two terms that often come up are "authentication" and "authorization." While they sound similar and are often used interchangeably, they serve distinct purposes in ensuring the security and integrity of software systems. Let's delve into the specifics of each and explore how they differ, along with some technologies commonly used to implement them.

Authentication: Who Are You?

Authentication is the process of verifying the identity of a user or entity attempting to access a system or resource. In simpler terms, authentication answers the question, "Who are you?" This process typically involves the user providing some form of credentials, such as a username and password, biometric data, security tokens, or digital certificates. The system then validates these credentials against stored records to determine whether the user is who they claim to be.

Technologies for Authentication:

Username and Password: This is the most common form of authentication, where users provide a unique username and a corresponding password.
Biometric Authentication: With advancements in technology, biometric authentication methods such as fingerprint scanning, facial recognition, iris scanning, and voice recognition are becoming increasingly popular for user authentication.
Security Tokens: Security tokens, such as smart cards or hardware tokens, generate one-time passwords or cryptographic keys that users must provide along with their other credentials.
OAuth 2.0: OAuth is an open-standard authorization protocol that allows users to access resources from one website using their credentials from another website or service without exposing their password. It is commonly used for granting third-party applications limited access to a user's resources without exposing their credentials.
Passwordless Authentication: Passwordless authentication eliminates the need for traditional passwords and relies on alternative methods such as magic links sent via email, SMS-based one-time passcodes (OTPs), or authentication apps. This approach enhances security and user experience by reducing the risk of password-related attacks such as phishing and credential stuffing, while also simplifying the login process for users.

Authorization: What Are You Allowed to Do?

Authorization, on the other hand, comes into play after authentication and determines what actions an authenticated user or entity is permitted to perform within the system. Authorization answers the question, "What are you allowed to do?" It involves defining and enforcing access controls based on the user's identity, role, or other attributes.

Authorization mechanisms specify the level of access granted to users for various resources or functionalities within the system. This could include read-only access, write access, administrative privileges, or custom permissions tailored to specific roles or individuals.

Technologies for Authorization:

OAuth 2.0: While OAuth is primarily an authentication protocol, it also has provisions for authorization through the use of access tokens. OAuth 2.0 enables third-party applications to access resources on behalf of a user with their consent, following a predefined authorization flow. Read, why Access Tokens are not sufficient for fine-grained access control.
Role-Based Access Control (RBAC): RBAC is a widely used authorization model that assigns permissions to roles rather than individual users. Users are then assigned one or more roles that determine their access rights within the system. RBAC simplifies access management by grouping users based on their job functions or responsibilities. This is, what Space Blocks Permissions is doing for fine-grained access control.
Attribute-Based Access Control (ABAC): ABAC defines access controls based on attributes associated with the user, the resource being accessed, and the environment. Policies are defined using attributes such as user roles, location, time of access, and other contextual information to make access decisions dynamically.
Access Control Lists (ACLs): ACLs are a mechanism for defining and enforcing access controls on individual resources. They specify which users or groups are granted access to a particular resource and what operations they are allowed to perform.

Don't build Authentication or Authorization yourself!

Implementing both, Authentication and Authorization on your own can be challenging and complex. You want to make sure, to follow all security requirements for storing user-passwords, for example.

This is, why most developers rely on external tools for Authentication and others for Authorization, which can then be included into their apps. Space Blocks Permissions is a system to integrate fine-grained access control quickly into your apps with a few lines of code, so developers can focus on their core business. If you need to add permissions to your app, give it a try with the free Developer Tier.

Conclusion

In summary, while authentication verifies the identity of users or entities accessing a system, authorization determines what actions they are allowed to perform once authenticated. Understanding the distinction between these two concepts is crucial for designing secure and robust software systems. By implementing appropriate authentication and authorization mechanisms using technologies such as those mentioned above, developers can ensure that their applications remain secure, and only authorized users have access to sensitive resources.

Google Zanzibar implementations

Robin-Manuel Thiel — Wed, 08 May 2024 05:22:00 +0000

Authentication is hard. Implementing fine-grained access control is even harder. Whoever tried to build a sophisticated authentication system themselves knows that. And those who didn't are well advised by not trying to. Proper Authentication is so hard, that 2020 Google published a research paper about the numerous challenges and complexity they faced when implementing a global permission system for YouTube, Google Drive and Google Photos. This research paper is called "Zanzibar".

What the paper is lacking are implementation details. And since its release, some brave developers tried to implement an Authentication system along Google's findings. Here is a list of Google Zanzibar implementations:

1. Space Blocks Permissions

Space Blocks has built the most complete implementation of the Google Zanzibar research paper. It supports hierarchical permission trees, inheritance of access rights and custom roles. It can be hosted on your own servers or in the Space Blocks Cloud, where it provides high-availability and short response times.

Pros:

Completeness: Most complete and feature-rich Zanzibar implementation.
Easy to integrate: Permissions checks are integrated with a few lines of code.
GUI: Graphical Editor for designing and visualizing permission trees.‍
Low Learning Curve: No proprietary modeling language. ****

Cons:

SDK Availability: Only for .NET, JavaScript and React. Other platforms need to use the REST-API
Additional API endpoint required: Your backend might need to implement additional API endpoints for your frontend

2. OpenFGA

Auth0’s OpenFGA project focuses on delivering a universal authorization system. The name stands for “Fine Grained Authorization,” emphasizing a granular approach to modeling authorization that can handle diverse use cases. It is owned by the Cloud Native Foundation.

Pros:

Granularity: Allows fine-grained control over permissions, catering to complex authorization scenarios.
Open-Source Community: Community contributions and improvements are encouraged.
Performance: OpenFGA aims for reliability and performance.

Cons:

Missing Features: Operation for Listing all items a user has access to is incomplete
Steep Learning Curve: Understanding the modeling language schema may require some effort.

‍

3. ORY Keto

An open-source implementation of the Google Zanzibar paper exists in the form of the ****ORY Keto project. The first working version was released, aiming to bring the concepts from the paper to practical use. It can be employed to manage permissions and roles in various applications and websites.

Pros:

Open-Source Community: Being open source, it benefits from community contributions and improvements.
Fast and Efficient: ORY Keto aims for high performance, ensuring rapid decision-making.

Cons:

Steep Learning Curve: The granular permission language might require some learning and understanding.

4. Permify

Permify is another open-source authorization service for creating and managing scalable authorization systems using fine-grained permissions. It draws inspiration from Google’s Zanzibar paper and offers various binding and crafting options, allowing engineers to work with performant, observable, and secure permission systems.

Pros:

Integration: Integrates into many Identity Providers like Okta and Azure Active Directory
Granularity: Allows fine-grained control over permissions, catering to complex authorization scenarios.
Open-Source Community: Benefits from community contributions.

Cons:

Steep Learning Curve: Understanding the modeling schema may require effort.

Run Large and Small Language Models locally with ollama

Robin-Manuel Thiel — Tue, 07 May 2024 09:37:30 +0000

Since ChatGPT, we all know at least roughly what Large Language Models (LLMs) are. You might have heard that they require an immense amount of GPU power to run. But did you know, there are also smaller and less powerful versions of some models (SLMs) that you can run locally on your computer?

Prerequisites

Download and install ollama
Download and install Docker

For this tutorial, we use ollama to download a model onto your machine and run it there. Despite you have an absolute power house with lots of GPU power in front of you, you might want to try with smaller models, called Small Language Models or SLMs like Llama 3 8B from Meta or Phi-3 Mini 3.8B from Microsoft.

Run a model locally with ollama

Before we can communicate with a model via ollama, we need to start ollama. The ollama app is basically just a small web server that runs locally on your machine and lets you communicate with the models (by default on http://localhost:11434).

Run the following command in the Terminal to start the ollama server.

ollama serve

Now we can open a separate Terminal window and run a model for testing. If the model you want to play with is not yet installed on your machine, ollama will download it for you automatically. You can find a full list of available models and their requirements at the ollama Library.

Run the following command to run the small Phi-3 Mini 3.8B model from Microsoft.

ollama run phi3

Now you can interact with the model and write some prompts right at the command line.

Chat with your own data locally

For the whole ChatGPT like experience, where we can also chat with our own data or web sources, we don’t just want to prompt the model through the terminal.

Luckily, there are some open-source projects like Open WebUI, which provide a web-based experience similar to ChatGPT, that you can also run locally and point to any model. To start the Open WebUI Docker container locally, run the command below in your Terminal (make sure, that ollama serve is still running).

docker run --rm -p 8080:8080 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

Now, you can open your browser at http://localhost:8080, and create an account. No worries, it all stays local, and you don’t even have to use a real E-Mail address.

Once logged in, you can choose a model in the top-bar of Open WebUI and start chatting.

Running Open WebUI locally connected to ollama
I would also like to encourage you to play with the Documents section, to perform some Retrieval Augmented Generation (RAG) with your local models!

Now enjoy playing with Small Language Models on your computer! 🎉

What is a JWT and why is it not sufficient for fine-grained access control

Robin-Manuel Thiel — Tue, 07 May 2024 08:46:00 +0000

While JWT (JSON Web Tokens) and session-based authentication mechanisms are robust tools for authenticating users and managing sessions, they fall short when it comes to fine-grained access control (FGA), which is where Space Blocks Permissions come into play.

Authentication vs. Authorization

In the realm of securing digital systems, two fundamental concepts often come into play: authentication and authorization. While authentication verifies the identity of a user, authorization dictates what actions they are allowed to perform within a system. So authentication is answering the question of whom the user is. Authorization answers the question of what the user is allowed to do.

How does JWT work?

JWT (JSON Web Token) is a digitally signed JSON document, which includes information about the current user. It usually gets issued to the user by the identity provider, after a user signed in successfully. With every request, the user then sends the JWT to the backend, which can then verify its authenticity at the identity provider and, in case of a positive outcome, decide to let the user do, what they want to do.

The information about the user in the JWT’s payload are called claims. The JWT Standard defines official claims, that a token can have, but any identity provider can also add custom claims. A JWT can hold any kind of information, but usually, you can find the following claims:

iss (issuer): Issuer of the JWT
sub (subject): Subject of the JWT (the user)
aud (audience): Recipient for which the JWT is intended
scope (scope): Actions for which the JWT is intended
exp (expiration time): Time after which the JWT expires
nbf (not before time): Time before which the JWT must not be accepted for processing
iat (issued at time): Time at which the JWT was issued; can be used to determine the age of the JWT

When it comes to authorization (checking, what a user is allowed to do), the aud (audience) and scope (scope) claims of a JWT are interesting.

By checking the audience, a backend service can check, if a caller, that presents a certain token should be allowed to communicate with a specific backend or service (the audiences).

When defining APIs, developers often add scopes like read:secrets or write:secrets to their routes. Depending on how the user acquired the Access Token and which permissions that user has, different scopes are listed in a JWTs claims. Backend services can check these scopes against the ones they defined as required for their APIs, to decide if a user is allowed to call a certain API method or not.

Why JWTs are insufficient for fine-grained access control

As you might have noticed in the example above, JWT claims like audience and scope can only be used to check, if a user can call a certain API method or interact with a certain service at all. It’s all or nothing. The read:secrets scope either allows a user to read secrets in your application or not.

The reality often looks different. In most applications, where users collaborate with and share resources among each other, they have different permissions on different resources. A user might have read and write permissions to all of their own folders, but only read permissions to those that got shared by a teammate. Some permissions can also imply access to other resources without explicitly mentioning them. Read access to a folder usually also implies access to the files and sub-folder within the shared folder.

JWTs are great and have their place in the authorization space, but should only be used to determine if a user should be let into a system and how far. When it comes to fine-grained access control on resource level, a more sophisticated approach like Space Blocks Permissions is needed.

Permissions as a Service with Space Blocks

With the Space Blocks Permissions service, you can offload the burden of managing permissions, roles, groups and inheritance to our Permissions as a Service system and focus on your core project. You can define the structure of your resources and their relationships, define permissions for them, create roles and then just let us know, whenever permissions got assigned or changed.

Your backend can still use JWT for basic authentication, but can check the Space Blocks API for fine-grained access control, once a user got let into your system.

You can integrate Space Blocks into your backend code with a few lines of code and completely for free in the Developer Tier.

The ultimate guide to Git Hooks

Robin-Manuel Thiel — Tue, 23 May 2023 09:24:00 +0000

What are Git Hooks, and why should I use them?

Git Hooks are a Git feature, which allows developers to execute scripts at certain points in their Git lifecycle, like before accepting a commit, before pushing to a remote repository or before accepting a commit message. Git Hooks can be used to "shift left" and automate tasks and enforce policies throughout the Git workflow.

Git Hooks live in the .git/hooks folder of a repository. Here, we can place scripts with the same name as the Git Hooks they should execute on (max. one per hook). Below, an example which checks the commit message for a certain pattern.

#!/bin/bash

MESSAGE=$(cat $1) 
COMMITFORMAT="^(feat|fix|docs|style|refactor|test|chore|perf|other)(\((.*)\))?: #([0-9]+) (.*)$"

if ! [[ "$MESSAGE" =~ $COMMITFORMAT ]]; then
  echo "Your commit was rejected due to the commit message. Skipping..."   
  exit 1
fi

Keep in mind, that each script in this folder has to be marked as executable by running chmod +x pre-commit-msg.

You can find more information and a list of available Git Hooks at githooks.com.

What to use them for?

Check commit messages: Especially useful, if you use conventional commits. Also, to avoid the famous "WIP" or "fix" commit messages.
Check coding style and Linters: A no-brainer. Nothing is more frustrating than going back to your code and fix that trailing space, after the CI pipeline failed because of it.
Check for secrets: (Optional) We should check our code for accidentally checked-in secrets, like passwords or connection strings.
Build and Test: (Optional) At least before pushing changes to a remote, we could make sure the code builds and the tests pass.

Problems

Can be bypassed

As Git Hooks can be bypassed. For example, with the --no-verfify flag for the git commit command. So we should never rely on their execution but rather see them as a helper to avoid frustration amongst developers, when they find their Pull Request checks failing.

As we can not rely on their execution, Git Hook checks should be repeated when checking the Pull Request on the server!

Not installed to each development environment by default

As Git Hooks live in the .git/hooks folder and the .git folder itself is not part of the version control, scripts that are placed into that folder will not be checked in. So new developers that are cloning the repository won't have the Git Hooks in their local .git folder. Also, remote environments, like GitHub's In-Browser editing or Dev Containers won't have the Git Hooks setup by default.

A common practice around this behavior is placing the scripts into a .githooks folder in the repository root, and then configuring Git to use its Hooks from there with the git config core.hooksPath .githooks command. This allows to check in the scripts into version control, but the command still needs to be run in every new environment and won't make the Hooks available by default on new machines or remote environments.

Dependencies might not be installed

Most scripts for checking coding style violations or commit message patterns aren't as basic as the one from above but use other tools like Prettier, ESLint or editorconfig-checker for Coding Style checks or Gitlint (highly recommended) for commit message policies. These tools might not be installed on new or remote environments.

Managing Git Hooks (and their dependencies) with pre-commit

To deal with the problems from above, there are many tools out there which make working with Git Hooks easier and more straight-forward. Amongst the Node.js and JavaScript community, Husky gained some popularity, as it natively integrates into the package.json file, where app dependencies and development dependencies are managed.

As I deal with projects across different programming languages and frameworks, I was looking for a more generic approach, which can be used with any project. The tool that convinced me the most is pre-commit, as it

manages and installs Git Hooks
manages and installs dependencies (= tools, that my Git Hooks use)
can be run in CI pipelines to check, if Git Hooks haven't been skipped
is technology independent (in contrast to popular Husky)

The configuration for pre-commit is stored in a .pre-commit-config.yaml file at the repository root and contains the hooks to configure and the tools that should be executed.

default_install_hook_types:
- pre-commit
- pre-push
- commit-msg

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
  rev: v4.0.1
  hooks:
  - name: Check, that no large files have been committed
    id: check-added-large-files

- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
  rev: '2.7.1'
  hooks:
  - name: Check EditorConfig
    id: editorconfig-checker
    alias: ec
    stages: ["pre-push"]

- repo: https://github.com/jorisroovers/gitlint
  rev: v0.19.1
  hooks:
  - name: Lint commit messages
    id: gitlint
  - id: gitlint-ci
    args: ['--commits', 'origin/main..HEAD']

Installing Git Hooks for each developer

The configuration above allows to manage Git Hooks and their dependencies centrally in the repository. But the pre-commit tool itself still needs to be installed on each machine. Once installed, the hooks (and their dependencies) can be installed with a single command.

pre-commit install

Cross-Checking Git Hooks in the CI pipeline

As Git Hooks can be bypassed, it is highly recommended to cross-check them in Pull Request Checks or the CI pipeline. This is another reason, why I like pre-commit so much: The scripts can be executed at-hoc for checking the current Git repository and its commits for violations.

With GitHub Actions for example, you can check Pull Requests with the following workflow.

name: PR Check
on:
  pull_request:

jobs:
  pr-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Install pre-commit
        run: pip install pre-commit

      - name: Check commits
        run: pre-commit run --hook-stage manual

Mitigate violations

But what to do, when Pre-Push or Pull Request checks find policy violations in our code or commit messages, but the commit is already created and part of the current branch's history?

In case of a committed file that needs to be adjusted, the history of the current branch can be soft-reset to undo all commits that happened on that branch. Don't worry, your work won't be lost. This will put all modifications back to the list of uncommitted changes.

git checkout pr_branch
git reset --soft main

Before creating a commit, we can now adjust the files. Afterward, we can create a new commit without the violation.

git add -A && git commit -m "commit message goes here"
git push --force

To change a commit message afterward, we can follow a comprehensive Guide from GitHub.

Both, resetting the history and changing commit messages is annoying. To avoid that, it is generally recommended to do most checks at the pre-commit level.