DEV Community: Claudiu Nicola

How to Use Private Go Modules with golangci-lint in GitHub Actions.

Claudiu Nicola — Fri, 05 Jan 2024 15:54:07 +0000

Intro

We're at a point in the software development evolution when everybody recognizes the importance of Continuous Integration (CI) in the Software Development Life Cycle - or at least I hope. To be honest, I can't remember a project I've worked on that didn't have at least a basic CI pipeline in place.

golangci-lint is an amazing open-source tool for CI in Go projects. Basically, it's an aggregator and a Go linters runner that makes life easier for developers. It includes all the well-known liners by default but also provides an easy way to integrate new ones.

============

Steps

In the following, we will go through the steps required for running golangci-lint via GitHub Actions for projects with private modules.

1. Setup personal access tokens in GitHub

That can be done in both, personal accounts and organizations - depending on where the code is.

You have to go to Settings -> Developer Settings and click on Generate new token button

In the page opened you can choose between a personal or an organisation account. Here depends on your use case. Is it the project inside your account or inside of an organization account?

Select the repository/repositories or even all the repositories (again, depending on your use case)

and grant Read-only access for Actions.

The access token will be shown. Copy it and keep it somewhere for the next step.

Now, let's go back to the repository where the GH Action should run. Go to Settings -> Security -> Secrets and variables

click on New repository secret, fill in the secret name, and paste the access token from the previous step

2. Setup GitHub Actions workflow

2.1. Add golangci-lint.yml to your workflows

If it is not already exists.

# source: https://github.com/golangci/golangci-lint-action?tab=readme-ov-file#how-to-use

name: golangci-lint
on:
  push:
    branches:
      - master
      - main
  pull_request:

permissions:
  contents: read
  # Optional: allow read access to pull request. Use with `only-new-issues` option.
  # pull-requests: read

jobs:
  golangci:
    name: lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v4
        with:
          go-version: '1.21'
          cache: false
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v3
        with:
          # Require: The version of golangci-lint to use.
          # When `install-mode` is `binary` (default) the value can be v1.2 or v1.2.3 or `latest` to use the latest version.
          # When `install-mode` is `goinstall` the value can be v1.2.3, `latest`, or the hash of a commit.
          version: v1.54

          # Optional: working directory, useful for monorepos
          # working-directory: somedir

          # Optional: golangci-lint command line arguments.
          #
          # Note: By default, the `.golangci.yml` file should be at the root of the repository.
          # The location of the configuration file can be changed by using `--config=`
          # args: --timeout=30m --config=/my/path/.golangci.yml --issues-exit-code=0 

          # Optional: show only new issues if it's a pull request. The default value is `false`.
          # only-new-issues: true

          # Optional: if set to true, then all caching functionality will be completely disabled,
          #           takes precedence over all other caching options.
          # skip-cache: true

          # Optional: if set to true, then the action won't cache or restore ~/go/pkg.
          # skip-pkg-cache: true

          # Optional: if set to true, then the action won't cache or restore ~/.cache/go-build.
          # skip-build-cache: true

          # Optional: The mode to install golangci-lint. It can be 'binary' or 'goinstall'.
          # install-mode: "goinstall"

2.2. Add GOPRIVATE and GH_ACCESS_TOKEN environment variables.

The first one is to let the workflow know the private dependencies and the second one is to specify the access token created above.

...
jobs:
  golangci:
    name: lint
    runs-on: ubuntu-latest
    env:
+     GOPRIVATE: github.com/claudiunicolaa
+     GH_ACCESS_TOKEN: ${{ secrets.GH_ACCESS_TOKEN }}
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v4
        with:
          go-version: "1.21"
          cache: false
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v3
...

2.3. Automatically include a GitHub access token in the URL when interacting with GitHub repositories

jobs:
  golangci:
    name: lint
    runs-on: ubuntu-latest
    env:
      GOPRIVATE: github.com/claudiunicolaa
      GH_ACCESS_TOKEN: ${{ secrets.GH_ACCESS_TOKEN }}
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-go@v4
        with:
          go-version: "1.21"
          cache: false
+     - run: git config --global url.https://$GH_ACCESS_TOKEN@github.com/.insteadOf https://github.com/
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v3

============

Wrap-up

In this article, we learned how to use golangci-lint in GitHub Actions for projects with private modules. We went through the steps required to set up personal access tokens in GitHub and then used them to create repository secrets. We also added the required environment variables and created a workflow to run golangci-lint via GitHub Actions. By following these steps, you can now use golangci-lint to automatically check your Go projects for common coding mistakes and ensure that your code is consistent and maintainable.

Thank you for reading this article. I hope that the information provided here will help you use golangci-lint in GitHub Actions for your Go projects with private modules.

All the examples can be found in this repository

Good luck with your software development journey!

NATS Message Deduplication in Go

Claudiu Nicola — Sat, 29 Apr 2023 00:32:47 +0000

Summary / TL;DR

natsdedup is a simple and efficient message deduplication solution for NATS-based applications, focusing on content-based deduplication.

It offers easy integration, configurable deduplication TTL, and in-memory caching.

Check out the GitHub repository to get started and contribute to its growth.

Repository: github.com/claudiunicolaa/natsdedup

——

Today, I would like to introduce you to natsdedup, a lightweight package for deduplicating messages on NATS subjects. This package comes in handy for users who are not utilizing NATS JetStream and its built-in deduplication feature.

The natsdedup package aims to enhance the efficiency and reliability of your NATS-based applications by offering a simple yet effective message deduplication solution. It listens to messages on a specified input subject, deduplicates them using a configurable time-to-live (TTL) cache, and forwards unique messages to a specified output subject.

An interesting aspect of natsdedup is that it focuses on deduplicating messages based on their content rather than their ID headers, unlike the built-in deduplication capabilities in NATS JetStream. This makes natsdedup a suitable alternative for scenarios where content-based deduplication is desired.

Let's dive into some key features of the natsdedup package:

Easy Integration: natsdedup is designed to be easily integrated into your existing NATS-based applications. Just create a new instance of the Deduplicator struct, configure it with the input and output subjects, and run the deduplication process.
Configurable Deduplication: The package allows you to set a custom deduplication time-to-live (TTL) value, giving you control over how long messages should be considered for deduplication.
In-Memory Caching: The current implementation uses an in-memory cache for temporarily storing messages and efficiently deduplicating them. This ensures low-latency processing and minimal overhead.
Concurrency Control: The package employs concurrency control mechanisms like mutexes to ensure thread-safe access to the shared message cache.
Extensibility: natsdedup has been designed with extensibility in mind. As the project evolves, there are plans to explore and implement more advanced caching strategies, such as disk-based caching, distributed caching, and support for popular caching systems like Redis or Memcached.

To get started with natsdedup, head over to the GitHub repository, where you will find usage examples, and the source code. Contributions are welcome, whether it's adding new features, improving existing ones, or providing feedback to help make this package even better.

In conclusion, natsdedup is a valuable addition to the NATS ecosystem, providing a straightforward message deduplication solution for applications that do not rely on NATS JetStream. Give it a try and see how it can improve the performance and reliability of your NATS-based projects.

Repository: github.com/claudiunicolaa/natsdedup

AWS Lambda Go logging middleware example

Claudiu Nicola — Fri, 26 Feb 2021 18:58:23 +0000

For experienced programmers: A recap is always welcomed.

For beginners: you will have a description of what are and how middlewares work in a Go web app. Don't forget, the comments section is open. Leave a comment if you need more explanations.

We will see what middlewares mean from a web application perspective.

What represents the middlewares in a web app?

Middleware is a way for defining the separation of concerns in an application (distributed or not). Sometimes is called "the glue" or "plumbing" since it helps "glue" together different services and passes data between them [1] [2]. Web applications are used for executing a set of instructions before the request is handle by a controller. The instructions are usually to filter the requests (e.g. is this user allowed to access this page?), logging the request and the list can continue.

How to implement a middleware in Golang?

Requirement: Implement a logging middleware for AWS Lambda.

Checkout this repository.

Inspired by: