DEV Community: Nelwhix

How to publish a Go package on Chocolatey

Nelwhix — Sat, 28 Sep 2024 13:41:02 +0000

In this article, I will cover how to publish a go package to Chocolatey. Chocolatey is one of the most popular package managers on windows and recently I wanted a windows build of my CLI published there but there were no online resources. After hours of digging through the docs, I was able to successfully publish here and in this guide I will be showing you how to publish yours as well.

Introduction

First we need to build a CLI to publish, if you have yours you can skip this. But to make this as simple as possible I will write a go program that prints the current system time. Create a main.go file and fill in these contents.

package main

import (
    "fmt"
    "time"
)

func main() {
    currentTime := time.Now()

    fmt.Println("Current time:", currentTime.Format("2006-01-02 15:04:05"))
}

Then to aid us publishing, we are going to use a wonderful package called Goreleaser. It allows publishing to Homebrew, Chocolatey and several other package managers as easily as possible. First we install goreleaser by running:

go install github.com/goreleaser/goreleaser/v2@latest

Then we scaffold the config file by running:

goreleaser init

This will create a .goreleaser.yaml file in the current working directory.

Now take a look at the builds section. Currently it will look something like this:

builds:
  - env:
      - CGO_ENABLED=0
    goos:
      - linux
      - windows
      - darwin

This section is for customizing the build and state what os, cpu arch you will like to create binaries for. For the full documentation, check here. But for the purpose of this. I will just add the name of the CLI and build for just windows since Choco is windows-specific

builds:
  -
    binary: currentTime
    env:
      - CGO_ENABLED=0
    goos:
      - windows

We will also add a chocolateys section. Here we will add all choco specific details. The full list of options is here. But the base to get your package published and approved by maintainers is this:

chocolateys:
  - name: currentTime
    title: {{ Package Title }}
    authors: {{ Your name }}}
    project_url: {{ Github Repo to the package }}
    url_template: "https://github.com/foo/bar/releases/download/{{ .Tag }}/{{ .ArtifactName }}"
    copyright: 2024
    license_url: https://github.com/foo/bar/blob/main/LICENSE
    project_source_url: https://github.com/foo/bar
    docs_url: {{ Link to Docs }}
    bug_tracker_url: https://github.com/foo/bar/issues
    release_notes: "https://github.com/foo/bar/releases/tag/v{{ .Version }}"
    api_key: "{{ .Env.CHOCOLATEY_API_KEY }}"
    source_repo: "https://push.chocolatey.org/"
    skip_publish: false

Fill these fields except url_template, source_repo, api_key(we will add this to a github action secret) and then we move on to the next step.

Creating a Github Action

Now we create a github action to publish this CLI to choco anytime we make a release. create a .github/workflows/release.yml file in your current working directory and add the following.

name: Build release binaries

on:
  workflow_dispatch:
  push:
    tags:
      - "*"

permissions:
  contents: write

jobs:
  goreleaser:
    runs-on: windows-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: 1.19

      - name: Verify Chocolatey installation
        run: choco --version

      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v5
        with:
          distribution: goreleaser
          version: ${{ env.GITHUB_REF_NAME }}
          args: release --clean
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          CHOCOLATEY_API_KEY: ${{ secrets.CHOCOLATEY_API_KEY }}

This workflow will be dispatched when a new tag is created on the repository.

It runs on a windows action
The windows action comes built with chocolatey, but we don't know if that can change in the future. so I added a step to verify that Choco is installed
Then we run goreleaser using the goreleaser action. The two environment variables GITHUB_TOKEN and CHOCOLATEY_API_KEY we will be adding in the next step.

Now push all your code to a github repository then let's proceed with the next step.

Adding Repository Secrets

Now our github workflow needs two secrets. GITHUB_TOKEN and CHOCOLATEY_API_KEY. github token is a github personal access token. You can get those in your settings > Developer settings on github. Then for the chocolatey api key. You go to the choco community page. sign up and then find your api key on the profile page. After that you will be adding them on the action secrets section of your repository settings page here.

For the next step, we will create a new github release.

First we create a tag on new publish
And then publish as a new release
After publishing, check the actions for the status of the workflow. If the workflow is successful, check you choco packages section to see that the package is published.

Conclusion

Thank you for following along with this. Happy building!

How to build a caching layer for your Laravel API

Nelwhix — Fri, 09 Aug 2024 19:58:08 +0000

Let's say you are building an API to serve some data, you discover GET responses are quite slow. You have tried optimizing your queries, indexing your database tables by frequently queried columns and you are still not getting the response times you want. The next step to take is to write a Caching layer for your API. 'Caching layer' here is just a fancy term for a middleware that stores successful responses in a fast to retrieve store. e.g. Redis, Memcached etc. then any further requests to the API checks if the data is available in the store and serves the response.

Prerequisites

Laravel
Redis

Before we start

I am assuming if you have gotten here, you know how to create a laravel app. You should also have either a local or cloud Redis instance to connect to. If you have docker locally, you can copy my compose file here. Also, for a guide on how to connect to the Redis cache driver read here.

Creating our Dummy Data

To help us see our caching layer is working as expected. of course we need some data let's say we have a model named Post. so I will be creating some posts, I will also add some complex filtering that could be database intensive and then we can optimize by caching.

Now let's start writing our middleware:

We create our middleware skeleton by running

php artisan make:middleware CacheLayer

Then register it in your app/Http/Kernel.php under the api middleware group like so:

    protected $middlewareGroups = [
        'api' => [
            CacheLayer::class,
        ],
    ];

But if you are running Laravel 11. register it in your bootstrap/app.php

->withMiddleware(function (Middleware $middleware) {
        $middleware->api(append: [
            \App\Http\Middleware\CacheLayer::class,
        ]);
    })

Caching Terminologies

Cache Hit: occurs when data requested is found in the cache.
Cache Miss: happens when the requested data is not found in the cache.
Cache Flush: clearing out the stored data in the cache so that it can be repopulated with fresh data.
Cache tags: This is a feature unique to Redis. cache tags are a feature used to group related items in the cache, making it easier to manage and invalidate related data simultaneously.
Time to Live (TTL): this refers to the amount of time a cached object stays valid before it expires. One common misunderstanding is thinking that every time an object is accessed from the cache (a cache hit), its expiry time gets reset. However, this isn't true. For instance, if the TTL is set to 5 minutes, the cached object will expire after 5 minutes, no matter how many times it's accessed within that period. After the 5 minutes are up, the next request for that object will result in a new entry being created in the cache.

Computing a Unique Cache Key

So cache drivers are a key-value store. so you have a key then the value is your json. So you need a unique cache key to identify resources, a unique cache key will also help in cache invalidation i.e. removing cache items when a new resource is created/updated. My approach for cache key generation is to turn the request url, query params, and body into an object. then serialize it to string. Add this to your cache middleware:

class CacheLayer 
{
    public function handle(Request $request, Closure $next): Response
    {
    }

    private function getCacheKey(Request $request): string
    {
        $routeParameters = ! empty($request->route()->parameters) ? $request->route()->parameters : [auth()->user()->id];
        $allParameters = array_merge($request->all(), $routeParameters);
        $this->recursiveSort($allParameters);

        return $request->url() . json_encode($allParameters);
    }

    private function recursiveSort(&$array): void
    {
        foreach ($array as &$value) {
            if (is_array($value)) {
                $this->recursiveSort($value);
            }
        }

        ksort($array);
    }
}

Let's go through the code line by line.

First we check for the matched request parameters. we don't want to compute the same cache key for /users/1/posts and /users/2/posts.
And if there are no matched parameters we pass in the user's id. This part is optional. If you have routes like /user that returns details for the currently authenticated user. it will be suitable to pass in the user id in the cache key. if not you can just make it an empty array([]).
Then we get all the query parameters and merge it with the request parameters
Then we sort the parameters, why this sorting step is very important is so we can return same data for let's say /posts?page=1&limit=20 and /posts?limit=20&page=1. so regardless of the order of the parameters we still return the same cache key.

Excluding routes

So depending on the nature of the application you are building. There will be some GET routes that you don't want to cache so for this we create a constant with the regex to match those routes. This will look like:

 private const EXCLUDED_URLS = [
    '~^api/v1/posts/[0-9a-zA-Z]+/comments(\?.*)?$~i'
'
];

In this case, this regex will match all a post's comments.

Configuring TTL

For this, just add this entry to your config/cache.php

  'ttl' => now()->addMinutes(5),

Writing our Middleware

Now we have all our preliminary steps set we can write our middleware code:

public function handle(Request $request, Closure $next): Response
    {
        if ('GET' !== $method) {
           return $next($request);
        }

        foreach (self::EXCLUDED_URLS as $pattern) {
            if (preg_match($pattern, $request->getRequestUri())) {
                return $next($request);
            }
        }

        $cacheKey = $this->getCacheKey($request);

        $exception = null;

        $response = cache()
            ->tags([$request->url()])
            ->remember(
                key: $cacheKey,
                ttl: config('cache.ttl'),
                callback: function () use ($next, $request, &$exception) {
                    $res = $next($request);

                    if (property_exists($res, 'exception') && null !== $res->exception) {
                        $exception = $res;

                        return null;
                    }

                    return $res;
                }
            );

        return $exception ?? $response;
    }

First we skip caching for non-GET requests and Excluded urls.
Then we use the cache helper, tag that cache entry by the request url.
we use the remember method to store that cache entry. then we call the other handlers down the stack by doing $next($request). we check for exceptions. and then either return the exception or response.

Cache Invalidation

When new resources are created/updated, we have to clear the cache, so users can see new data. and to do this we will tweak our middleware code a bit. so in the part where we check the request method we add this:

if ('GET' !== $method) {
    $response = $next($request);

    if ($response->isSuccessful()) {
        $tag = $request->url();

        if ('PATCH' === $method || 'DELETE' === $method) {
            $tag = mb_substr($tag, 0, mb_strrpos($tag, '/'));
        }

        cache()->tags([$tag])->flush();
    }

    return $response;
}

So what this code is doing is flushing the cache for non-GET requests. Then for PATCH and Delete requests we are stripping the {id}. so for example if the request url is PATCH /users/1/posts/2 . We are stripping the last id leaving /users/1/posts. this way when we update a post, we clear the cache of all a users posts. so the user can see fresh data.

Now with this we are done with the CacheLayer implementation. Lets test it

Testing our Cache

Let's say we want to retrieve all a users posts, that has links, media and sort it by likes and recently created. the url for that kind of request according to the json:api spec will look like: /posts?filter[links]=1&filter[media]=1&sort=-created_at,-likes. on a posts table of 1.2 million records the response time is: ~800ms

and after adding our cache middleware we get a response time of 41ms

Great success!

Optimizations

Another optional step is to compress the json payload we store on redis. JSON is not the most memory-efficient format, so what we can do is use zlib compression to compress the json before storing and decompress before sending to the client.
the code for that will look like:

$response = cache()
            ->tags([$request->url()])
            ->remember(
                key: $cacheKey,
                ttl: config('cache.ttl'),
                callback: function () use ($next, $request, &$exception) {
                    $res = $next($request);

                    if (property_exists($res, 'exception') && null !== $res->exception) {
                        $exception = $res;

                        return null;
                    }

                    return gzcompress($res->getContent());
                }
            );

        return $exception ?? response(gzuncompress($response));

The full code for this looks like:

<?php

namespace App\Http\Middleware;

use Closure;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\Response;

class CacheLayer
{
    private const EXCLUDED_URLS = [];

    public function handle(Request $request, Closure $next): Response
    {
        $method = $request->getMethod();

        if ('GET' !== $method) {
            $response = $next($request);

            if ($response->isSuccessful()) {
                $tag = $request->url();

                if ('PATCH' === $method || 'DELETE' === $method) {
                    $tag = mb_substr($tag, 0, mb_strrpos($tag, '/'));
                }

                cache()->tags([$tag])->flush();
            }

            return $response;
        }

        foreach (self::EXCLUDED_URLS as $pattern) {
            if (preg_match($pattern, $request->getRequestUri())) {
                return $next($request);
            }
        }

        $cacheKey = $this->getCacheKey($request);

        $exception = null;

        $response = cache()
            ->tags([$request->url()])
            ->remember(
                key: $cacheKey,
                ttl: config('cache.ttl'),
                callback: function () use ($next, $request, &$exception) {
                    $res = $next($request);

                    if (property_exists($res, 'exception') && null !== $res->exception) {
                        $exception = $res;

                        return null;
                    }

                    return gzcompress($res->getContent());
                }
            );

        return $exception ?? response(gzuncompress($response));
    }

    private function getCacheKey(Request $request): string
    {
        $routeParameters = ! empty($request->route()->parameters) ? $request->route()->parameters : [auth()->user()->id];
        $allParameters = array_merge($request->all(), $routeParameters);
        $this->recursiveSort($allParameters);

        return $request->url() . json_encode($allParameters);
    }

    private function recursiveSort(&$array): void
    {
        foreach ($array as &$value) {
            if (is_array($value)) {
                $this->recursiveSort($value);
            }
        }

        ksort($array);
    }
}

Summary

This is all I have for you today on caching, Happy building and drop any questions, commments and improvements in the comments!