DEV Community: Aashish Koshti

Launched my iOS app - Tuneo Offline Music Player

Aashish Koshti — Fri, 08 Aug 2025 13:06:19 +0000

🚀 Just released my iOS app!

🎧 It’s called Tuneo — a clean, ad-free offline music player for iPhone.

I built it for myself because I couldn’t find a good way to play my own M4As, MP3s without ads or login walls.

Now it’s live on the App Store, and I’d love for you to try it!

📱 Import your own audio files
🎶 Edit tracks, manage playlists
📴 100% offline + runs in the background
🚫 No ads, no tracking, no login

App store: https://apps.apple.com/us/app/tuneo-offline-music-player/id6748915689

Website: https://tuneo.arcade.build/

Please consider following me on my socials.

X / Twitter: @ArcadeBuilds
Github: @the-arcade-01

Building With Docker: Images, Containers, Multi-Stage builds & Optimization

Aashish Koshti — Wed, 09 Jul 2025 16:07:49 +0000

In this blog, I'll cover how you create a Docker Image, run & manage Containers, What are Multi-Stage builds, few optimization tricks and much more with a simple & practical Golang app example.

Prerequisites.

No as such prerequisites required apart from the following:

You've Docker install on your system.
You're are familiar with few basic CLI commands (though they will be covered again in this blog).

Plus, you can also read my previous blog on Docker Basics: Beginner's Introduction.

Understanding Image Layers & Caching.

As a recap from my previous blog, Docker Images are templates which contains the instructions for creating a Docker Container. We first build the Image and use that to create a Container.

The instructions are written step-by-step and are called as Layers in Docker terms.

Images are by default immutable, which means, if we make changes in any of the Layers a new Image will get created rather than the old Image getting updated.

The Layers are stacked on top of each other, meaning they follow Top to Bottom down approach of execution. Each Layer depends on the previous Layer, meaning if you made any changes on the previous Layer then all the Layers below it will be rebuild again.

Since only the changed Layer and the Layers below it are rebuilt, Docker caches and stores the previous unchanged layers to speed up subsequent builds. This is called Layer Caching.

In the example above, Layer 3 was changed, hence all the below layers got rebuild but not the previous layers. Also a new Image got created.

In many production applications, we have multiple steps for building an Image. To take an example, let's say our app has multiple development environments, Production, Staging, Local. But all of these environments have the same starting Layers.

So, we can club those Layers together and use them as a starting point for other environments. This is also called as Builder Layer or Base Image depending on the context. Builder Layer if its used as a starting Layer for creating multiple environment Images for the same app, and Base Image if its used as a starting Image for creating multiple Images for different apps.

We'll look into Builder Layer in Multi-Stage builds section of this blog.

Writing a DockerFile.

We talked about Layers in the previous section. These Layers are written inside a special file which is called a DockerFile.

To explain how to write a DockerFile, we will use a simple Golang app example.

Note: No Golang experience is required. You can follow along with the explanation and use that knowledge to any of your apps.

This is the folder structure and inside which we have placed our DockerFile.

.
├── Dockerfile
├── go.mod
└── main.go

A simple program which starts a http server on port 8080 and when you hit endpoint / will return the Hello, Docker with Go! message.

package main

import (
    "fmt"
    "net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, "Hello, Docker with Go!")
}

func main() {
    http.HandleFunc("/", handler)
    fmt.Println("Server running on port 8080...")
    http.ListenAndServe(":8080", nil)
}

Response from the API.

> curl -X GET http://localhost:8080/
Hello, Docker with Go!

Now, let's talk about the DockerFile. What all Layers we will write to create an Image for our app.

DockerFile follows a specific syntax and each Layer starts with an Instruction specifier like FROM, CMD, COPY etc. You can check the full list of instruction over here Official DockerFile Reference.

# Layer 1: Base Image
FROM golang:1.24-alpine

# Layer 2: Set environment variables
ENV GO111MODULE=on \
    CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64

# Layer 3: Set working directory
WORKDIR /app

# Layer 4: Copy & install dependencies
COPY go.mod ./
RUN go mod download

# Layer 5: Copy source code
COPY . .

# Layer 6: Build binary file
RUN go build -o main

# Expose port
EXPOSE 8080

# Run the binary file
CMD ["./main"]

Now, let's understand each of the Layers & Instruction in detail.

Layer 1: Base Image
```
 FROM golang:1.24-alpine
```
Since, using the DockerFile we will create a Container which are Isolated. So, we require a building block, a Base image, on top of which we construct our app.

Here, we've used the official golang alpine image which is present on the Docker Hub. This Image will give us all the necessary golang related binaries & libraries for running our app.

We use the FROM instruction for using the base image.
Layer 2: Set environment variables

To set any app specific environment variables we can use the ENV <key>=<value> instruction.
Layer 3: Set working directory

WORKDIR instruction is used for changing the directory in which we would like to execute the below layers. We simply do WORKDIR <folder_name>.
Layer 4: Copy & install dependencies
```
COPY go.mod ./
RUN go mod download
```
COPY <src> <dest> lets you copy the files from your current directory to working directory.
Layer 5: Copy source code we copied all the files of our folder to the working directory.
Layer 6: Build binary file

RUN <cmd> using this instruction we can run any command which we want to be performed inside the working directory.
Expose port

EXPOSE <port> using this we tell Docker that our container will listen to this port. This step doesn't creates a Layer.
Run the binary file
```
CMD ["./main"]
```
CMD instruction is used for specifying what command you need to run when starting the Container. This step doesn't creates a Layer.

An important difference between CMD & RUN,

RUN executes the specified command at the Build time of the Image,

CMD executes the specified command at the Container Run time.

The above Instruction are very commonaly used inside a DockerFile and you will use these quite a lot when building your own DockerFile.

Building Image & Running Containers.

Now, let's run few commands to build our Image.

To build our Image, we run this command

docker build .

The . specifies the build context. Since, we are in the same directory as of our DockerFile we don't need to change the build context. After running this we will get the output.

You read the output and see what all steps have been executed.

If we re-run the command, you'll see that all the steps were Cached because we didn't made any changes.

To see all the Images in our system, we can run this command

docker images

It returned us the Image of our app with an ID. We can use the ID to start the container but it will be difficult to find which app is ours if you build multiple images.

For that we can Tag our image with some name. We run the build command like this

docker build -t <name> ./

Now, if your run docker images you will see our Image got new name myapp.

Now, let's run the Container using our Image. For that we use the following command

docker run <image_name or image id>

Nice, our app is running let's test the API by hitting the curl

curl -X GET http://localhost:8080/

Hey, but we got this as a response.

We got this response because we didn't mount our system port 8080 to the container's exposed port. For that we run the following command

docker run -p <system_port>:<container_port> <image_name or image id>

Now, hit the API again and you will see this response.

Now, here are some more commands which you can run and play with.

docker ps to get list of all running containers.
docker ps -a to get list of all the containers.
docker stop <container_id> stops the container.
docker remove <container_id> to delete a container.
docker rmi <image_id> deletes the image.
docker exec -it <container_id> <cmd> execute cmd inside the container

You don't need to memorize all these commands, you can just look them up here Official CLI Docs.

Multi-Stage builds.

If we look into our previous example of our DockerFile, we'll find that we can divide our DockerFile primarily in two separate operations.

First being a Build & Compile step & second being the Execution step.

Now, if we look at the size of our myapp image, we will see its quite large for such a simple app.

To reduce & optimize the construction of our Images, Docker provides us with a functionality of Multi-Stage builds using which we can divide our DockerFile into multiple Stages such that the final Image will be of lower size.

To understand this, let's take the previous example and convert it into Multi-stage build.

# Stage 1: Build
FROM golang:1.24-alpine AS builder

ENV CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64

WORKDIR /app

COPY go.mod ./
RUN go mod download

COPY . .

RUN go build -o main

# Stage 2: Final image (runtime only)
FROM alpine:latest

WORKDIR /app

COPY --from=builder /app/main .

EXPOSE 8080

CMD ["./main"]

Now, let's understand both of these Stages in detail.

Stage 1: Build

FROM instruction marks the beginning of a Stage using the AS <stage_name>.

In this first stage, we are creating the binary file of our Golang app.
Stage 2: Final image

In this stage we are using a lightweight Base image alpine for just executing our binary file.

COPY --from=<stage_name> <source> <dest>, here we are specifying that we only need the binary file to be copied into the final image.

How this works is?

When you run the build command, Docker will sequentially execute the stages but only the final Stage will be created as an Image.
The builder stages will still remain in the Cached state and will be used for subsequent builds.

Now, let's run the build command & check the size of the image.

docker build -t mymultistageapp ./

We observed a significant reduction in the size of our image.

Using Multi-Stage builds in our DockerFile we ensure that:

Our Images are lightweight & optimized in size.
DockerFile is split efficiently in multi stages.
Only necessary files are shipped into the final Image.

Optimizing Images & Best Practices.

Few of things which we can do to make are Images efficient & follow best practices are:

Using .dockerignore for excluding unnecessary files.

We can create a .dockerignore file for excluding unwanted files.
```
    *.log
    *.env
    .git
    vendor/
    node_modules/
```
Using Multi-Stage builds and correctly spliting DockerFile into distinct Stages.
Rebuilding Images & defining Layers correctly for leveraging Build Cache.

You can refer to more of these here Official Docs for Best practices.

Summary.

In this blog, we covered a lot of important concepts for building and working with Docker Images. In upcoming blogs, we will see more advanced topics like Docker Compose, Working example of a Fullstack (3-Tier architecture) app on Docker, Networking, Volumes and much more.

So, keep an eye on this series Docker In Depth.

Conclusion.

Thanks for reading it, and if you liked the blog, please consider following me on my socials.

X / Twitter: @ArcadeBuilds
Github: @the-arcade-01

Docker Basics: Beginner's Introduction

Aashish Koshti — Thu, 03 Jul 2025 15:12:38 +0000

In this blog, I'll walk you through Docker basics, architecture, and important concepts like Image, Containers with a simple & practical example.

History of Developing & Deploying application.

In past, we used to deploy applications on host system (a simple machine) & ran them directly on it.

A host machine has a fixed Operating system, softwares, libraries etc, installed on it and we were able to develop and deploy applications on them.

But there were certains issues like dependencies conflicts, software conflicts, data corruption etc. Imagine you are running two applications on the same machine both of them access the same database and one of them fails and corrupts the entire DB. Because of this we sought for new solutions.

Then came a concept of Virtualization, which helped managing and distributing the resources of a host system in much more efficient manner.

The idea of a Virtual Machine came. A Virtual Machine (VM) in simple terms can be understood as a small host system running inside another host system, which has its own Operating System, Kernel, binaries, applications etc.

By using VM, we were able to create multiple host systems inside our main host system and deployed our applications efficiently.

But as time passed, we realized that managing these VM were still a time consuming process. Imagine, your application accidently gets 1 million users traffic and now you need to scale it up. But to prepare a new VM on a host system will take time, because it's a whole OS (just a smaller size) and setting it up takes a lot of time. Think of the time when you updated your windows, or installed linux on your machine.

Because of these downsides, a simple solution was needed.

And then came a new concept called Containers which Docker uses.

Containers proposed a new way of managing the application environments. Instead of all the bloated VM which contains their own OS, binaries, Kernels etc, Containers said, instead of dividing our host system to cater our application, why don't we package our application and it's dependencies together and then run them on the system.

This allowed developers to run as many packaged apps, called as Containers, on their host system efficiently. This overall improved the whole development to deployment cycle in many companies.

By using Containers, we were able to isolate the applications into their own individual environments. This solved the problem of shared resources between the applications as well as managing & deploying these containers were much easier than VM.

Docker Architecture.

Docker uses the philosophy of Containers to provide developers better experience of developing & deploying applications. It provides us a whole tooling for managing the lifecycle of our Containers.

Let's look at the architecture diagram of Docker.

Docker Daemon listens and performs action on the API requests given to it by the client.
Docker Client provides CLI CMDs using which we can communicate with the Docker Daemon.
Docker Registries stores Docker images. Docker Hub is a public registry on which you can find multiple software images like golang, node js etc.

In most cases, we Developers don't need to go in-depth about Docker architecture & how it's build. We are more concerned about the core things which it provides us for working with containers.

Image, Containers & Registry.

Most important things we as Developers need to know about are Images, Containers & Registry. So, let's look into them.

Images

An Image is a blueprint or a template which contains the necessary steps for creating a container.

To give an example, let's take Burger. To make a burger, you take buns, lettuce, tomato, ketchup, patty, and place all of them in layers to form a burger. Similarly, to build a container, in which your application will run. You specify all the necessary steps inside an Image. These steps are also called as Layers.

To build a Image, we use DockerFile. Inside this file we write instructions. We'll work with Images in more detail in upcoming blogs.

Containers

A Container on the other hand is a runnable instance of an Image. Using the Image as a template, you create the Container. You can create, start, stop, etc. and perform multiple actions on it.

Containers are isolated, meaning they don't interfere with other Containers running on the same host system. We can also provide configuration options as per our requirements if we want multiple Containers to work together. We'll work with Containers in more detail in upcoming blogs.

Registry

A Registry is a place which stores Images. Docker Hub is public registry on which you can find many images for different softwares.

If you want to run a software on your machine, instead of installing it, you get it's Image from the public registry and run it as a Container on your machine. This removes the overhead of installing softwares on our machine.

Some CLI CMDs & Practical Example.

Now, let's look at some of the commands which Docker provides us using a simple example. By the way, follow these instructions for installing Docker on your system. Official Installation Docs.

Suppose you want to run a simple Nginx web server using Docker. Here are the basic steps:

docker pull

This command downloads an image from Docker Hub to your local machine.

docker pull nginx

docker images

Lists all Docker images available on your local machine.

docker images

docker run

Runs a container from the specified image. The following command starts an Nginx container and maps port 8080 on your machine to port 80 in the container.

docker run -d -p 8080:80 --name mynginx nginx

-d runs the container in detached mode (in the background).
-p 8080:80 maps port 8080 on your host to port 80 in the container.
--name mynginx gives your container a name.

docker ps

Lists all running containers.

docker ps

Now, if you open your browser and go to http://localhost:8080, you should see the default Nginx welcome page running from your Docker container.

Summary.

This was a very basic introduction to Docker, but in upcoming blogs I'm going to cover a lots of important topics like Building efficient Images, Docker Networking, Storage, Multi Stage builds, CI/CD Actions and many more.

So, keep an eye on this series Docker In Depth.

Conclusion.

Thanks for reading it, and if you liked the blog, please consider following me on my socials.

X / Twitter: @ArcadeBuilds
Github: @the-arcade-01

AWS Cognito Authentication with Golang

Aashish Koshti — Fri, 02 May 2025 02:30:00 +0000

In this blog, I've covered practical implementation of AWS Cognito in your Golang application.

Code can be found in this github repo:
https://github.com/the-arcade-01/golang-aws-cognito-auth

Pre-Requisites

You need to have AWS Account & access keys set on your system.
You have experience in building APIs in Golang.

AWS Cognito User Pool Setup

To perform user management and authentication/authorization, we will use Cognito's User Pool service, which basically is a user directory, stores users and gives us functionality to manage them.

Lets start with the AWS Cognito setup on the console.

Go to the AWS Cognito - User Pool page, choose your AWS region on the top.
Now we will create the User pool.

For the type of application, our app is server-side Golang API app. So, choose the first option, i.e, Traditional Web Application.
Now, in the Configure options, we will set Email as an identifier. You can select other identifiers as well (phone and username), using which user will be able to signup/signin in the app. We also have attributes to select, attributes are aditional information that you want from your user, like his name, address, country etc. We will pick the name option.
Now, go ahead and create the user pool by clicking on the Create User Directory button.

Now go to the User Pool page and open your created user pool and copy the Token Signing Key URL & User Pool ID and paste them in your .env file.
Now go to the App Clients page and from here you need to copy the Client ID & Client Secret and paste it in your .env file.
Now click on the Edit button on top right and then in the Authentication Flows enable the ALLOW_USER_PASSWORD_AUTH option & save the changes.

That's it, after the setup you will have the following variables in your .env file. And don't worry I'm gonna delete this app, so I don't mind sharing the values xD.

AWS_COGNITO_USER_POOL_ID=ap-south-1_zCnGOgkNP
AWS_COGNITO_CLIENT_ID=6af3ihrq538f4a17a4mf9o38vn
AWS_COGNITO_CLIENT_SECRET=j7ptvlppl1hqh5mr0pr4ngh5ovfbld1ouulef86ope9h014pgik
AWS_COGNITO_TOKEN_URL=https://cognito-idp.ap-south-1.amazonaws.com/ap-south-1_zCnGOgkNP/.well-known/jwks.json
AWS_COGNITO_JWT_ISSUER_URL=https://cognito-idp.ap-south-1.amazonaws.com/ap-south-1_zCnGOgkNP

Code Explanation

Now, for the code, I'm going to explain the important parts of the app only. For the full code you can check the repo.

First, I've parsed the .env variables into a Config and also initialized an aws.Config object.

type Config struct {
    Env                    string
    Port                   string
    AwsCognitoUserPoolId   string
    AwsCognitoClientId     string
    AwsCognitoClientSecret string
    AwsConfig              aws.Config
    AwsTokenURL            string
    AwsJWTIssuerURL        string
}

... // inside Load config func
    awsCfg, err := awsconfig.LoadDefaultConfig(context.Background())
    if err != nil {
        return nil, fmt.Errorf("failed to load AWS configuration")
    }

    cfg := &Config{
        Env:                    v.GetString("ENV"),
        Port:                   v.GetString("PORT"),
        AwsCognitoUserPoolId:   v.GetString("AWS_COGNITO_USER_POOL_ID"),
        AwsCognitoClientId:     v.GetString("AWS_COGNITO_CLIENT_ID"),
        AwsCognitoClientSecret: v.GetString("AWS_COGNITO_CLIENT_SECRET"),
        AwsConfig:              awsCfg,
        AwsTokenURL:            v.GetString("AWS_COGNITO_TOKEN_URL"),
        AwsJWTIssuerURL:        v.GetString("AWS_COGNITO_JWT_ISSUER_URL"),
    }
...

Now, I've defined the Cognito store which exposes all the necessary functions, which we will look one by one later. Also, we are using latest aws-sdk-go-v2 version.

type CognitoStore struct {
    client       *cognitoidentityprovider.Client
    userPoolId   string
    clientId     string
    clientSecret string
    tokenURL     string
    jwtIssuerURL string
    jwkSet       jwk.Set
}

I've created the cognito client and also I'm fetching the JWT Key Set from the AwsTokenURL. We need this because, when we create a JWT Token, AWS signs that token with its private key. So, in order to check whether the token is legit or not, we fetch Public Keys set from the token URL and then use that to verify the tokens signature. We will look at this later in ValidateToken function.

func NewCognitoStore(cfg *config.Config) (*CognitoStore, error) {
    keySet, err := jwk.Fetch(context.Background(), cfg.AwsTokenURL)
    if err != nil {
        return nil, fmt.Errorf("failed to fetch JWK set: %w", err)
    }
    return &CognitoStore{
        userPoolId:   cfg.AwsCognitoUserPoolId,
        clientId:     cfg.AwsCognitoClientId,
        clientSecret: cfg.AwsCognitoClientSecret,
        tokenURL:     cfg.AwsTokenURL,
        jwtIssuerURL: cfg.AwsJWTIssuerURL,
        client:       cognitoidentityprovider.NewFromConfig(cfg.AwsConfig),
        jwkSet:       keySet,
    }, nil
}

Now, lets look at the SignUp function. At the time of User Pool creation we selected Email as the identifier, so in the SignUpInput object, we pass user.Email in the Username (its as per AWS doc & yes its confusing xD). And we also had additional name attribute which we selected at the time of creation. So we pass that in UserAttributes.

...
type User struct {
    Name     string `json:"name"`
    Email    string `json:"email"`
    Password string `json:"password"`
}
...
func (s *CognitoStore) SignUp(ctx context.Context, user *models.User) error {
    input := &cognitoidentityprovider.SignUpInput{
        ClientId: aws.String(s.clientId),
        Password: aws.String(user.Password),
        Username: aws.String(user.Email),
        UserAttributes: []types.AttributeType{
            {Name: aws.String("name"), Value: aws.String(user.Name)},
        },
        SecretHash: aws.String(s.generateSecretHash(user.Email)),
    }

    _, err := s.client.SignUp(ctx, input)
    if err != nil {
        slog.ErrorContext(ctx, "Failed to sign up user", "email", user.Email, "err", err)
        return appError.NewServiceUnavailableError("Unable to process registration")
    }

    return nil
}
...

Also, you can see I've passed a secret hash value, it basically is an encoded value using the Client Secret for your app. It checks whether your app has the right to use the User Pool or not.

func (s *CognitoStore) generateSecretHash(username string) string {
    h := hmac.New(sha256.New, []byte(s.clientSecret))
    h.Write([]byte(username + s.clientId))
    return base64.StdEncoding.EncodeToString(h.Sum(nil))
}

Now, lets check the ConfirmAccount function. When a user registers in User Pool, AWS automatically sends Verification code to the user email by default. You can configure the settings on this page.

Since, we didn't configure anything regarding this. Our users will get the verification code. The ConfirmSignUpInput takes in the identifier (which in our case is Email) and the code.

...
type UserConfirmationParams struct {
    Email string `json:"email"`
    Code  string `json:"code"`
}
...
func (s *CognitoStore) ConfirmAccount(ctx context.Context, user *models.UserConfirmationParams) error {
    _, err := s.client.ConfirmSignUp(ctx, &cognitoidentityprovider.ConfirmSignUpInput{
        ClientId:         aws.String(s.clientId),
        ConfirmationCode: aws.String(user.Code),
        Username:         aws.String(user.Email),
        SecretHash:       aws.String(s.generateSecretHash(user.Email)),
    })

    if err != nil {
        slog.ErrorContext(ctx, "Failed to confirm account", "email", user.Email, "err", err)
        return appError.NewServiceUnavailableError("Unable to confirm account")
    }

    return nil
}
...

Now, lets check the Login function. The InitiateAuthInput takes in the Email and Password and the secret hash. On successful response it returns us a result object, from which we are only concerned about AccessToken, RefreshToken & Access token ExpiresIn time.

func (s *CognitoStore) Login(ctx context.Context, user *models.UserLoginParams) (*models.AuthLoginResponse, error) {
    output, err := s.client.InitiateAuth(ctx, &cognitoidentityprovider.InitiateAuthInput{
        AuthFlow:       types.AuthFlowTypeUserPasswordAuth,
        ClientId:       aws.String(s.clientId),
        AuthParameters: map[string]string{"USERNAME": user.Email, "PASSWORD": user.Password, "SECRET_HASH": s.generateSecretHash(user.Email)},
    })

    if err != nil {
        slog.ErrorContext(ctx, "Failed to sign in user", "email", user.Email, "err", err)
        return nil, appError.NewServiceUnavailableError("Authentication service unavailable")
    }

    authResult := output.AuthenticationResult
    if authResult == nil {
        return nil, appError.NewServiceUnavailableError("Invalid authentication result")
    }

    return models.NewAuthLoginResponse(
        aws.ToString(authResult.AccessToken),
        aws.ToString(authResult.RefreshToken),
        int(authResult.ExpiresIn),
    ), nil
}

Now, to fetch an User details, only thing we need is the AccessToken.

func (s *CognitoStore) GetUser(ctx context.Context, token string) (*models.UserInfoResponse, error) {
    output, err := s.client.GetUser(ctx, &cognitoidentityprovider.GetUserInput{
        AccessToken: aws.String(token),
    })

    if err != nil {
        slog.ErrorContext(ctx, "Failed to get user info", "err", err)
        return nil, appError.NewServiceUnavailableError("Unable to fetch user info")
    }

    attributes, username := output.UserAttributes, output.Username
    if attributes == nil || username == nil {
        return nil, appError.NewServiceUnavailableError("Invalid user info result")
    }

    attributesMap := make(map[string]string)
    for _, attribute := range attributes {
        attributesMap[aws.ToString(attribute.Name)] = aws.ToString(attribute.Value)
    }

    return &models.UserInfoResponse{
        Attributes: attributesMap,
        Username:   aws.ToString(username),
    }, nil
}

So, we are done with the API functions which we will expose to the client from our app.
Now, lets look at the JWT Middleware functions.

First one is the ValidateToken function. As per AWS Doc for Verifying A JWT Token - User Pool, I've followed the same steps in the code.
You can also check we are using JWT KeySet to check whether the key ID is present in our set or not.

func (s *CognitoStore) ValidateToken(tokenString string) (*jwt.Token, error) {
    token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
        if _, ok := token.Method.(*jwt.SigningMethodRSA); !ok {
            return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
        }

        kid, ok := token.Header["kid"].(string)
        if !ok {
            return nil, errors.New("key ID not found in token")
        }

        key, found := s.jwkSet.LookupKeyID(kid)
        if !found {
            return nil, errors.New("key not found in JWKS")
        }

        var rawKey interface{}
        if err := key.Raw(&rawKey); err != nil {
            return nil, fmt.Errorf("failed to get raw key: %w", err)
        }

        return rawKey, nil
    })
    if err != nil {
        return nil, fmt.Errorf("invalid token: %w", err)
    }

    claims, ok := token.Claims.(jwt.MapClaims)
    if !ok || !token.Valid {
        return nil, errors.New("invalid token claims")
    }

    issuer, err := claims.GetIssuer()
    if err != nil {
        return nil, errors.New("token has invalid issuer")
    }

    if strings.Compare(issuer, s.jwtIssuerURL) != 0 {
        return nil, errors.New("token was not issued by the specified Cognito user pool")
    }

    return token, nil
}

func (s *CognitoStore) GetClaims(token *jwt.Token) (map[string]interface{}, error) {
    claims, ok := token.Claims.(jwt.MapClaims)
    if !ok {
        return nil, errors.New("invalid token claims")
    }
    return claims, nil
}

Now, lets define a JWT Auth Middleware which we will use infront of private APIs. Here, we took the Token from the headers. Validated it using our ValidateToken function in CognitoStore Auth store, then get the user claims and passed them into request context along with the token.

func jwtAuthMiddleware(authStore db.AuthStore) func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            header := r.Header.Get("Authorization")
            if header == "" {
                models.ResponseWithJSON(w, http.StatusUnauthorized, models.NewErrorResponse(http.StatusUnauthorized, "Authorization header required"))
                return
            }

            parts := strings.Split(header, " ")
            if len(parts) != 2 || parts[0] != "Bearer" {
                models.ResponseWithJSON(w, http.StatusUnauthorized, models.NewErrorResponse(http.StatusUnauthorized, "Invalid authorization header format"))
                return
            }

            token, err := authStore.ValidateToken(parts[1])
            if err != nil {
                models.ResponseWithJSON(w, http.StatusUnauthorized, models.NewErrorResponse(http.StatusUnauthorized, "Invalid authorization token "+err.Error()))
                return
            }

            userInfo, err := authStore.GetClaims(token)
            if err != nil {
                models.ResponseWithJSON(w, http.StatusInternalServerError, models.NewErrorResponse(http.StatusInternalServerError, "Failed to extract user info"))
                return
            }

            ctx := context.WithValue(r.Context(), models.RequestContextKey, &models.RequestContext{
                UserInfo: userInfo,
                Token:    parts[1],
            })
            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }
}

Below code is how we use the JWT Auth Middleware to protect our routes. The /user/info API will require access token in headers.

...
    authRouter.Post("/signup", authHandlers.SignUp)
    authRouter.Post("/login", authHandlers.Login)
    authRouter.Post("/confirm", authHandlers.ConfirmAccount)

    authRouter.Group(func(r chi.Router) {
        r.Use(jwtAuthMiddleware(authStore))
        r.Get("/user/info", authHandlers.GetUser)
    })
...

API Testing

First, lets hit the /auth/signup API.

curl --location 'http://localhost:8080/auth/signup' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "Bunty",
    "email": "bunty@deliveryotter.com",
    "password": "Bunty@123"
}'

in response we got this.

{
  "status": 201,
  "data": {
    "message": "User registered successfully."
  }
}

We will also receive an email containing the verification code. Which will look like this.

Now, lets hit the /auth/confirm API.

curl --location 'http://localhost:8080/auth/confirm' \
--header 'Content-Type: application/json' \
--data-raw '{
    "email": "bunty@deliveryotter.com",
    "code": "882246"
}'

in response we got this.

{
  "status": 200,
  "data": {
    "message": "Account confirmed successfully."
  }
}

You can also check the User in the AWS Console Users page.

Now, lets test the /auth/login API.

curl --location 'http://localhost:8080/auth/login' \
--header 'Content-Type: application/json' \
--data-raw '{
    "email":"bunty@deliveryotter.com",
    "password": "Bunty@123"
}'

in response will get receive access_token, refresh_token & expires_in.

{
  "status": 200,
  "data": {
    "access_token": "<access_token>",
    "refresh_token": "<refresh_token>",
    "expires_in": 3600
  }
}

Now, hit the /auth/user/info API with the AccessToken passed in the headers.

curl --location 'http://localhost:8080/auth/user/info' \
--header 'Authorization: Bearer <access_token>'

in response we got,

{
  "status": 200,
  "data": {
    "attributes": {
      "email": "bunty@deliveryotter.com",
      "email_verified": "true",
      "name": "Bunty",
      "sub": "31133daa-00d1-70f1-4227-b5422184d374"
    },
    "username": "31133daa-00d1-70f1-4227-b5422184d374"
  }
}

Conclusion

Thanks for reading it.
Please follow me on my socials:

References

Golang LLD: Design a Task Scheduler

Aashish Koshti — Fri, 04 Apr 2025 05:15:15 +0000

In this blog post, I've shared my approach to solve this low level design problem, Design a Task Scheduler.

Code for this can be found on my Github repo:
https://github.com/the-arcade-01/golang-low-level-design/tree/main/task-scheduler

Requirements

Users should be able to create task, schedule task & stop task.
Users can schedule task of type
- One Time: execute task only once
- Fixed Rate: execute task at every recurring delay
- Fixed Delay: execute task at every recurring delay but after the its previous run completion.
Task execution should be based on priority and execution time.
Multiple task should execute in concurrently, take care of application's CPU and memory usage.
Assume a task execution to be a function call, eg: task.Run() simply prints task details.

Thought Process

Thinking about the entities that will be involved.
Task entity will have all the information about a task.
Tasks needs to be executed based on priority & execution time. So, some kind of data structure for maintaining the order.
We choose Heap.
Why?
Heap offers us O(logn) time complexity for insert & deletion operation while maintaining the correct order. Whereas a traditional sort on list/slice will be of O(nlogn) and we need to perform sort after each insert & deletion step.
For concurrent execution of tasks while managing CPU & memory, we will use Worker Pool pattern.
This will allow us to open n fixed go routines which will listen to a worker queue for incoming task.
For executing the task, we will have to run a separate go routines whose job will be to pick task from the task heap and put them into the worker pool.

Code structure

Code structure of our app.

.
├── cmd
│   └── main.go
├── go.mod
├── internal
│   ├── config
│   │   └── pool.go
│   ├── models
│   │   └── runnable.go
│   └── task
│       ├── heap.go
│       ├── scheduler.go
│       ├── service.go
│       └── task.go

Worker pool is defined in config folder. Worker pool takes in a runnable interface instance for executing a Run() (we will look that later).
All task related functionality is present inside task folder.

Code overview

First we define the worker pool. A worker pool has a no. of workers & a worker queue (in which task are put in). And this worker pool will be a Singleton structure. Because this will load as a configuration in our system.

For making it singleton, we will use sync.Once.

var (
    once sync.Once
    pool *WorkerPool
)

type WorkerPool struct {
    workers     int
    workerQueue chan models.Runnable
    wg          sync.WaitGroup
}

Next step in here is to spawn workers no. of go routines which will listen to the worker queue.

func NewWorkerPool(workers int) *WorkerPool {
    once.Do(func() {
        pool = &WorkerPool{
            workers:     workers,
            workerQueue: make(chan models.Runnable),
        }

        for i := range pool.workers {
            pool.wg.Add(1)
            fmt.Printf("Worker routine initialized, ID: %v\n", i)

            go func(i int) {
                defer pool.wg.Done()

                for run := range pool.workerQueue {
                    run.Run()
                }
            }(i)
        }
    })
    return pool
}

The Runnable interface looks like this, our Task will implement the Run() function.

type Runnable interface {
    Run()
}

Now moving to the heap for the tasks. To make a heap we need to implement to sets of properties
First is sort methods, in which we write our ordering logic.

type TaskHeap []*Task

// sort methods
func (h TaskHeap) Len() int { return len(h) }
func (h TaskHeap) Less(i, j int) bool {
    if h[i].ExecutionTime.Compare(h[j].ExecutionTime) == 0 {
        return h[i].Priority < h[j].Priority
    }
    return h[i].ExecutionTime.Before(h[j].ExecutionTime)
}
func (h TaskHeap) Swap(i, j int) {
    h[i], h[j] = h[j], h[i]
}

& second is the heap methods.

// heap methods
func (h *TaskHeap) Push(x any) {
    *h = append(*h, x.(*Task))
}
func (h *TaskHeap) Pop() any {
    old := *h
    n := len(old)
    item := old[n-1]
    old[n-1] = nil
    *h = old[0 : n-1]
    return item
}

Now moving to the TaskScheduler, it will also be a Singleton structure as I've used it as a single entry point for scheduling & executing of the task.

type TaskScheduler struct {
    mtx       *sync.Mutex
    cond      *sync.Cond
    pool      *config.WorkerPool
    taskQueue *TaskHeap
}

Schedule function will look like this, takes in a Task and we perform heap insertion on the task queue.

func (t *TaskScheduler) Schedule(task *Task) {
    t.mtx.Lock()
    defer t.mtx.Unlock()

    fmt.Printf("Task: %v Scheduled\n", task.Name)
    heap.Push(t.taskQueue, task)
    t.cond.Signal()
}

Stop function will look like this.

func (t *TaskScheduler) Stop(task *Task) {
    t.mtx.Lock()
    defer t.mtx.Unlock()

    for i, item := range *t.taskQueue {
        if item == task {
            fmt.Printf("Task: %v Stopped\n", task.Name)
            heap.Remove(t.taskQueue, i)
            break
        }
    }
}

Now the Execute() function. Its task is to take the Task out of the task queue and put that into the worker pool.
To implement this behavior we've to approaches:

Brute Force approach:
- Run a ticker (a simple cron) every 1 sec or so.
- Check the size of queue and the top element, if valid for execution then push it in worker pool.
- Pros: simple and straightforward to execute.
- Cons: running a ticker is unnecessary if there are no task to perform. This in total increase the cpu consumption and the tasks doesn't get executed properly on their correct time.
Optimizing the process
- For this we will use a signal that will notify our Execute() func go routine, that, hey! a new task has been inserted!! wake up, check whether you need to put it in pool or not. If not, then go back to sleep again till its execution time.
- Using this we ensure that our go routine, is only working when its require to.
- To use signal in Golang, we use sync.Cond, it helps us to define condition on which we need to signal our go routine.
- If the task queue is empty, we do wait t.cond.Wait().
- For signalling the go routine, we do that in the Schedule(task *Task) function at end using this t.cond.Signal().

func (t *TaskScheduler) Execute() {
    for {
        t.mtx.Lock()
        if t.taskQueue.Len() == 0 {
            t.cond.Wait()
        }

        task := heap.Pop(t.taskQueue).(*Task)
        if task.ExecutionTime.After(time.Now()) {
            heap.Push(t.taskQueue, task)

            t.mtx.Unlock()
            time.Sleep(time.Until(task.ExecutionTime))
            continue
        }

        t.mtx.Unlock()
        t.pool.Add(task)
    }
}

Now the Run() function which our Task structure will implement.
In this func, we will do the execution of task and further scheduling as per TaskType.

func (t *Task) Run() {
    if t.Type == FixedRate {
        t.UpdateExecutionTime(time.Now().Add(time.Duration(t.RecurringDelay) * time.Second))
        taskScheduler.Schedule(t)
    }

    // performing execution of the task, simple print statement
    t.execute()

    if t.Type == FixedDelay {
        t.UpdateExecutionTime(time.Now().Add(time.Duration(t.RecurringDelay) * time.Second))
        taskScheduler.Schedule(t)
    }
}

Conclusion

Thanks for reading it.
Please follow me on my socials:

And if you've any suggestions or find any bugs in my code or want to share your own approach, feel free to open an Issue on my github Repo with the format below:

Title: "Approach: <Problem Name>"
Example: "Approach: Design a Cache System"

Understanding Authentication & Authorization (JWT, OAuth2.0, Session) Concepts

Aashish Koshti — Thu, 03 Apr 2025 08:13:56 +0000

In this blog, I've shared my understanding on different Auth strategies.

What is Authentication & Authorization?

Authentication: The process of verifying the identity of a user or system. It ensures that the entity requesting access is who they claim to be. Common methods include passwords, biometrics, and tokens.
Authorization: The process of determining what an authenticated user or system is allowed to do. It defines the permissions and access levels granted to the authenticated entity, such as reading, writing, or deleting resources.

Auth Strategies

Basic Authentication

Most simplest form of authentication in which the client sends a Base64 encoded string in the Authorization Basic Header which is made from user credentials.
An example would be, <email>:<password>, this is the format used and then encode this and attach it in the header.
Eg:

Field	Value
Credentials	example@example.com:1234
Base64 encoded	ZXhhbXBsZUBleGFtcGxlLmNvbToxMjM0
Header	Authorization: Basic ZXhhbXBsZUBleGFtcGxlLmNvbToxMjM0

The server then receives the request, decodes the string, matches them against the database values and serves the response.
Pros: Simple to use, can be used in services which don't require high level security.
Cons: Doesn't provide extensive security, encoded string can be easily decrypted.

JWT Authentication

JWT or JSON Web Tokens are token-based strategy which allows parties to securely transmit information between themselves using JSON objects.
The JWT is transferred in an encoded format which has 3 parts:
Header
- Header contains the information about the cryptographic algorithm used and the type of token.
- eg:

   {
      "alg": "HS256",
      "typ": "JWT"
   }

Payload
- A JSON object which contains the info the parties wanted to send between each other. Also known as Claims.
- There are some standard fields like "iat" (issued at), "exp" (expire at) etc, which are necessary for JWT validation and ensure security but are subject to your use case.
- eg:

   {
      "sub": "1234567890",
      "name": "John Doe",
      "admin": true,
      "iat": 1516239022
   }

Signature
- It's an encrypted string generated after applying the cryptographic algorithm in combination with the header, payload, and a secret key.
- A secret key is a private key known only to the server, used to sign the token and ensure its integrity.
- The signature ensures that the token has not been altered. If any part of the token changes, the signature will no longer match, and the token will be considered invalid.
- Eg. of a signature which uses SHA 256 algorithm:

      HMACSHA256(
      base64UrlEncode(header) + "." +
      base64UrlEncode(payload),
      secret)

Eg. of a JWT:

  <Base64Url-encoded header>.<Base64Url-encoded payload>.<Base64Url-encoded signature>

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMn0.KMUFsIDTnFmyG3nMiGM6H9FNFUROf3wh7SmqJp-QV30

Verification and Validation of JWT:
- Verification involves checking the token's signature to ensure it was generated by the server and has not been tampered with.
- Validation involves checking the token's claims, such as expiration time, to ensure the token is still valid.
How JWT auth flow works?
- Access Token:
  - An access token is a short-lived token that is used to access protected resources. It is issued by the authentication server after the user successfully authenticates.
  - The access token is included in the Authorization header of HTTP requests to the server.
  - Access tokens have a limited lifespan and expire after a certain period, requiring the user to obtain a new token.
- Refresh Token:
  - A refresh token is a long-lived token that is used to obtain a new access token without requiring the user to re-authenticate.
  - The refresh token is issued along with the access token and is stored securely on the client side most likely in HTTP-Only secure cookie.
  - When the access token expires, the client can use the refresh token to request a new access token from the authentication server.
  - Refresh tokens also have an expiration time, but it is usually much longer than that of access tokens.
  - Most auth systems store refresh tokens in a database to provide additional security and revoke capabilities if it gets stolen.
- JWT Auth Flow:
  1. The client sends a login request with user credentials to the authentication server.
  2. The server verifies the credentials and, if valid, issues an access token and a refresh token.
  3. The client stores the tokens securely and includes the access token in the Authorization header of subsequent requests to access protected resources.
  4. When the access token expires, the client uses the refresh token to request a new access token from the server.
  5. The server verifies the refresh token and, if valid, issues a new access token.
  6. The client continues to use the new access token to access protected resources.
Pros:
- Stateless: JWTs are self-contained and do not require server-side storage, making them ideal for distributed systems.
- Scalability: Since JWTs are stateless, they can be easily scaled across multiple servers without the need for session synchronization.
- Flexibility: JWTs can carry custom claims, allowing for flexible and extensible authentication mechanisms.
Cons:
- Token Size: JWTs can become large, especially when carrying many claims, which can impact performance.
- Security: If not properly secured, JWTs can be vulnerable to attacks such as token theft or tampering.
A Practical implementation of JWT Authentication in Golang: https://github.com/the-arcade-01/golang-jwt-authentication

OAuth 2.0 Authorization

OAuth 2.0 (Open Authorization) is an authorization strategy that allows third-party applications to access user resources without exposing user credentials.
Example: Let's say in our app we want to access a user's photos which are stored in Google Drive. In this case, our app needs to access those photos on behalf of the user. To do that, we implement a mechanism using Google Auth provider, in which we specify to the user that we will only require permission to access the photos. Once the user grants us permission, we:

Ask the Google Auth provider to generate an access token.
Use the access token to access the Google photos.

Using this flow, we do not require the user to share their Google credentials and do not need to manage any additional information. Our app only needs client_id & client_secret for communicating with the Google Auth provider.

A simple OAuth 2.0 flow might look like this:
Pros:
- Security: OAuth 2.0 allows users to grant limited access to their resources without sharing their credentials, reducing the risk of credential theft.
- Granular Permissions: OAuth 2.0 supports fine-grained access control, allowing users to specify the exact permissions they want to grant to third-party applications.
- User Experience: OAuth 2.0 provides a seamless user experience by allowing users to authenticate using their existing accounts with trusted providers.
Cons:
- Complexity: Implementing OAuth 2.0 can be complex due to the various flows and security considerations involved.
- Token Management: Managing access and refresh tokens securely requires careful handling to prevent token theft and misuse.
- Dependency on Third-Party Providers: Relying on third-party authentication providers can introduce dependencies and potential points of failure.
A Practical implementation in Golang & React.js:
https://github.com/the-arcade-01/react-golang-oauth-flow

Session-Based Authentication

In Session-Based Authentication, a unique ID (known as "session ID") is generated on the server and is stored into a database after a successful login of the users. This identifier is then send backed to the user, and is stored in Cookies at the client side.
How Session-Based auth flow works?

User performs a login with its valid credentials.
The server validates those credentials and then generates a unique ID, stores that into some database (eg: Redis) with an expiration time (TTL, Time To Live), and then sends it back to the user.
The client then stores that session ID in the cookies and calls the subsequent requests using that ID.
When server receives a request with the session ID, it fetches the session ID from database, checks whether it has not expired, then compares it with the request one, and if valid then proceed otherwise not.
When user logs out then session ID is deleted. Plus system also use expiration time on Session ID so they get automatically deleted from the database.

Pros: Easier to implement, provides expiry control (is used redis then automatic expiry), session ID is stored in database which is secured.
Cons: Good for monolithic apps, for microservices, it requires centralized storage and service so the system is consistent like using API Gateway for all entry points and performing auth check.

Conclusion

Thanks for reading it and I hope I explain the concepts in an easy way. If you've any comments or want to give any suggestion, please leave it in the comments section.

Also I didn't cover RBAC (Role-Based Access Control) & ABAC (Attribute-Based Access Control) authorization methods because they are centric towards access control model. Although you can read about them here:

References

Some of the resources which I've used for writing this blog, these are some great resources which will help you understand things in more details.

Golang LLD: Design a Cache System (LRU, LFU, FIFO)

Aashish Koshti — Sat, 29 Mar 2025 14:22:16 +0000

In this blog post, I've shared my approach to solve this low level design problem, Design a Cache system.

Code for this can be found on my Github repo:
https://github.com/the-arcade-01/golang-low-level-design/tree/main/cache-system

Requirements

Our cache should support:

Basic operations:
- Put(key, value, ttl): Add a key-value pair with a time-to-live (TTL).
- Get(key): Retrieve the value for a key if it exists and hasn't expired.
- Delete(key): Remove a key manually.
Eviction Policies:
- LRU (Least Recently Used)
- LFU (Least Frequently Used)
- FIFO (First In First Out)
- Evict entries when the cache reaches its maximum capacity.
TTL Expiry:
- Automatically remove keys that have expired based on their TTL.
Code should be extensible and adhere best practices.

Code structure

First of all, defining the structure of the code. I follow this folder structure:

├── cmd
│   └── main.go
├── go.mod
└── internal
    ├── cache
    │   └── cache.go
    ├── evictionpolicy
    │   ├── evictionpolicy.go
    │   ├── fifo.go
    │   ├── lfu.go
    │   └── lru.go
    └── models
        └── node.go

cmd entry point for our main app.
cache folder, we define the cache struct and its operations.
evictionpolicy (could be named better xD), contains interface declaration with its implementations.
models, contains common models.

Now, the classes (or I should say structs, as Golang formally don't have OOP definitions)

Cache struct holds and expose all the public methods which are mentioned in requirements.

EvictionPolicy, since we've multiple algorithms which we can implement, so our code should be extensible and allow us to choose which algorithm we would like to have in our cache. Hence, we created an interface holding all the methods which are necessary in defining the algorithms.

In a system we can have different instances of cache running with different algorithms. This is an example of Factory Pattern.
Using a constructor we can create different eviction policies.

func NewEvictionPolicy(policy PolicyType) (EvictionPolicy, error) {
    switch policy {
    case LRU:
        return newLRUCache(), nil
    case FIFO:
        return newFIFOCache(), nil
    case LFU:
        return newLFUCache(), nil
    }

    return nil, fmt.Errorf("policy not found")
}

Node is just a data model which represents a key-value pair,

Code explanation

Before diving into the code part, if you're not familiar with the algorithms mentioned above, please read about them then it will be easier for you to understand the code, though I'll explain my things are required but I won't explain all the algorithms. Plus a basic understanding of Generics is also required.

Our Cache storage first of all has a variable capacity which tells us the maximum count of keys which it can store. Then we've the eviction policy which decides which logic we apply to remove the keys, when the size of cache exceeds the capacity. We also have storage map which stores the keys to their respective data pointers. This gives us O(1) lookup & insert.

Why are we using *list.Element as a type for the value?

Because the implementation of the algorithms mentioned above requires a doubly linked list to maintain the specific order of the elements. Eg: in LRU algorithm, the elements which are least recently used will be present at the bottom end of the list, while the ones which are frequently used will be towards the front.

type Cache[T comparable, U any] struct {
    mtx            sync.Mutex
    capacity       int
    expireInterval int
    storage        map[T]*list.Element
    evictionpolicy evictionpolicy.EvictionPolicy
}

Now, let's look at the individual methods exposed by the Cache struct.

GET method logic
1. If the key is present in the map, then we perform the re-ordering of that element in the eviction list (eg: in case of LRU, the element is moved to front, in case of LFU, its frequency is increased and is moved to new frequency list).
2. Before returning we also check whether its expired or not which is a simple time check node.ttl.Before(time.Now()).
3. If not found, then we return a zero value with an error not found.

func (c *Cache[T, U]) Get(key T) (U, error) {
    c.mtx.Lock()
    defer c.mtx.Unlock()

    if element, ok := c.storage[key]; ok {
        element = c.evictionpolicy.Get(element)
        c.storage[key] = element
        if node, ok := element.Value.(*models.Node[T, U]); ok {
            if !node.IsExpire() {
                return node.GetValue(), nil
            }
        }
    }
    var zeroValue U
    return zeroValue, fmt.Errorf("key not found")
}

Put method logic
1. If the key is present in the storage, then we update its value and TTL and then again perform the re-ordering of the element.
2. If the key is not present, in that case, first we check if the size of cache is full or not. If full, then we need to perform eviction based on the policy (eg: in case of LRU, the last element of the list is removed, in case of FIFO, the first element is removed). We delete the element from both storage map and eviction list.
3. Then we insert in new key value in storage.

func (c *Cache[T, U]) Put(key T, value U, ttl int) {
    c.mtx.Lock()
    defer c.mtx.Unlock()

    if c.capacity == 0 {
        return
    }

    if element, ok := c.storage[key]; ok {
        element.Value.(*models.Node[T, U]).UpdateValue(value).UpdateTTL(ttl)
        element = c.evictionpolicy.Get(element)
        c.storage[key] = element
    } else {
        if c.capacity <= len(c.storage) {
            del := c.evictionpolicy.Evict()
            if del != nil {
                delete(c.storage, del.Value.(*models.Node[T, U]).GetKey())
            }
        }
        c.storage[key] = c.evictionpolicy.Put(models.NewNode(key, value, ttl))
    }
}

Delete method We delete the given key from both the eviction list and storage map.


func (c *Cache[T, U]) Delete(key T) {
    c.mtx.Lock()
    defer c.mtx.Unlock()

    c.delete(key)
}

func (c *Cache[T, U]) delete(key T) {
    if element, ok := c.storage[key]; ok {
        c.evictionpolicy.Delete(element)
        delete(c.storage, key)
    }
}

Now, let's look at the eviction policy interface and LRU cache implementation.

type EvictionPolicy interface {
    Get(element *list.Element) *list.Element
    Evict() *list.Element
    Put(node any) *list.Element
    Delete(element *list.Element)
}

Get method is responsible for re-ordering the element to its correct position.
Evict contains logic for evicting the element if the size of cache reaches its limit.
Put inserts the data into the list.
Delete I guess deletes the element, xD.

An example of LRU implementation looks like this,

type LRUCache struct {
    list *list.List
}

func (c *LRUCache) Get(element *list.Element) *list.Element {
    c.list.MoveToFront(element)
    return element
}

func (c *LRUCache) Evict() *list.Element {
    last := c.list.Back()
    if last != nil {
        c.list.Remove(last)
    }
    return last
}

func (c *LRUCache) Put(node any) *list.Element {
    return c.list.PushFront(node)
}

func (c *LRUCache) Delete(element *list.Element) {
    if element != nil {
        c.list.Remove(element)
    }
}

Further improvements

Optimizing the TTL based keys removal

Currently we loop through all the keys in map and check their TTL which is $O(n) time complexity and is not efficient.

func (c *Cache[T, U]) expire() {
  ticker := time.NewTicker(time.Duration(c.expireInterval) * time.Second)
  fmt.Printf("Cache expire interval started, interval: %v\n", c.expireInterval)

  for range ticker.C {
      c.mtx.Lock()
      for key, value := range c.storage {
          if value.Value.(*models.Node[T, U]).IsExpire() {
              c.delete(key)
          }
      }
      c.mtx.Unlock()
  }
}

We might optimize this by maintaining a heap of (keys, TTL), this way we won't have to loop for all the keys, and only those keys will be removed which have expired, just need to periodically check the top of the heap, time complexity of heapify $O(logn).

Storage map is initialized in Cache struct itself which can be moved out as a separate interface.

Conclusion

Thanks for reading it, and if you've any suggestions or find any bugs in my code or want to share your own approach, feel free to open an Issue on my github repo with the format below:

Title: "Approach: <Problem Name>"
Example: "Approach: Design a Cache System"

Or you can leave any comments here.
Thanks for reading it :).

Using Redis to build a Distributed Chat App in Go & React.js

Aashish Koshti — Sun, 02 Mar 2025 18:41:56 +0000

In this blog, I've covered my approach on building a simple but scalable distributed chat application using Golang, Redis & Websockets.

Code for this:
https://github.com/the-arcade-01/react-golang-distributed-chat-app

You can test the app itself here: https://chat.arcade.build

Theory

First of all, what does `Distributed` even means?

Distributed in terms of system architecture means multiple components (whether it be server, database, message brokers etc) communicates with each other, even though they are running on their own machines (or nodes), instead of all being handled by a single machine.

In our context, we have separate database (redis) and backend server (golang websocket) running on their own machines.

Why it is good though? Since, each component serves their own purpose, managing them is a lot more easier. Let's take an example, if our backend server is not able to handle loads of websocket connections, then we can quickly spin up multiple instances.

Now, let's talk about `Websockets`

Websockets are communication protocol, which allows full duplex communication (both client & server can send messages) over a single TCP connection.

How they work?

First the client sends a HTTP request to the server having header Upgrader as websocket.
Then the server responds with a HTTP 101 status code which means switching protocols.
Once the handshake is completed, both client and server can transfer data.

One more important thing in websockets are the ping-pong mechanism check, which tells us whether the connection is still active between client and server. How it works is, the server on some periodic intervals sends ping message to the client and client responds with a pong message. These messages are useful because they indicate that the connection is still active between other the resources and at any moment if one of the operation fails then we can disconnect the connection and reestablish it.

`Redis` & How we are using `Streams` and `PubSub` for our chat.

Redis offers us two solutions which serves our purpose, i.e, chat communication, via Streams and PubSub.

Redis `PubSub`

Basically acts as a simple channel, in which you can send message (Publish operation) and receives message (Subscribe operation)
Has a simple concept of Fire & Forget, means messages are not stored and they will only be delivered to the user if he was subscribed to channel before the message was sent.
There can be multiple Publishers and multiple Subscribers.

Redis `Streams`

Streams acts as an append only log, which stores the incoming messages and allows us to perform different retrival operations on them like range query etc.
Since message are stored in stream, we've the functionality to send those messages to new subscribers (like sending chat history or previous messages).
Each event or message in a stream have timestamp field attached to it for maintaining the order.
We can also have multiple publishers and multiple subscribers to a stream.

Code Overview

Here, I won't paste all the code from the repo. I'll just explain few implementation details which are necessary for the working.

Backend code

In my Golang server, I've used Gorilla's websocket library for handling websocket server. It's a very popular and battle tested library and easy to use with great documentation.

Establishing the websocket connection by upgrading the HTTP request, we first define the websocket upgrader and then in the handler we use it to upgrade the HTTP connection to a websocket connection.

upgrader: websocket.Upgrader{
        ReadBufferSize:  1024,
        WriteBufferSize: 1024,
        CheckOrigin: func(r *http.Request) bool {
            origin := r.Header.Get("Origin")
            return origin == Envs.WEB_URL
        },
    },
... 

func (h *handlers) handleWS(w http.ResponseWriter, r *http.Request) {
    username := r.URL.Query().Get("username")
    if username == "" {
        http.Error(w, "Username is required", http.StatusBadRequest)
        Log.ErrorContext(r.Context(), "Username is required")
        return
    }

    conn, err := h.upgrader.Upgrade(w, r, nil)
    if err != nil {
        Log.ErrorContext(r.Context(), "WebSocket upgrade failed", "error", err)
        http.Error(w, "Failed to upgrade connection", http.StatusInternalServerError)
        return
    }

    Log.InfoContext(r.Context(), "WebSocket connection established", "username", username)

    ctx := context.Background()
    go h.writePump(ctx, conn)
    go h.readPump(ctx, conn, username)
}

We then open two goroutines, one which listens to the user's messages via ws and publishes then into the message channel (readPump), other which takes the messages from the redis message channel and sends them back to the user via ws conn (writePump).
Now, I've implemented both the approaches PubSub and Streams separately, so let's look into them

PubSub:

Redis makes creating of the pubsub channel easier, we just need to call the Publish method on the name of the channel, if it exist then message will be pushed, if not then redis will first create it and then will push the message.

  ... 
  // in readPump func
    for {
      // listening to user's input message
          _, msg, err := conn.ReadMessage()
          if err != nil {
              if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseAbnormalClosure) {
                  Log.ErrorContext(ctx, "error on ws", "error", err)
              }
              break
          }
      // publishing that message into redis pubsub, the command looks like this
          if err := h.cacheRepo.publish(ctx, Envs.CHAT_CHANNEL, string(msg)); err != nil {
              Log.ErrorContext(ctx, "error publishing message to redis", "error", err)
          }
      }

Now, since this is a simple channel, in the writePump function we just open this channel and listen to the incoming messages, and sends them back to the user
Here we also took care of the ping-pong mechanism check, which helps us to remove any stale connections or any inactive connections (stale conn with > 30sec of inactivity)

    func (h *handlers) writePump(ctx context.Context, conn *websocket.Conn) {
        ticker := time.NewTicker(pingPeriod)
        pubsub := h.cacheRepo.redis.Subscribe(ctx, Envs.CHAT_CHANNEL)

        defer func() {
            pubsub.Close()
            ticker.Stop()
            conn.Close()
        }()

        ch := pubsub.Channel()

        for {
        select {
        case <-ctx.Done():
          return
        case msg, ok := <-ch:
          if !ok {
            conn.WriteMessage(websocket.CloseMessage, []byte{})
            return
          }

          // Expliciting checking message size, even though checkOrigin check is added
          // so that any manuall attempt to direct conn on ws with the same origin
          // doesn't cause cause conn break
          if len(msg.Payload) > maxMessageSize {
            Log.ErrorContext(ctx, "message size exceeds limit", "size", len(msg.Payload))
            continue
          }

          // if message writing takes more time than writeWait,
          // which can means client as slow internet
          // and then just hang the conn and UI will restablish it
          conn.SetWriteDeadline(time.Now().Add(writeWait))
          if err := conn.WriteMessage(websocket.TextMessage, []byte(msg.Payload)); err != nil {
            Log.ErrorContext(ctx, "error sending messsage to websocket", "error", err)
            return
          }
        case <-ticker.C:
          // Here we are pinging the client on pingPeriod to check whether conn exists
          // this is done to remove any stale conns
          conn.SetWriteDeadline(time.Now().Add(writeWait))
          if err := conn.WriteMessage(websocket.PingMessage, nil); err != nil {
            Log.ErrorContext(ctx, "error sending ping message", "error", err)
            return
          }
        }
      }
    }

Streams:

Both the writePump and readPump functions are the same only the part where pubsub is used is being changed and instead streams code is used.

First we need to initialize the stream, though redis allows us to initialize the stream just like the pubsub channel (if not present then create it), but here I've initialized it before so that I maintain only 1 stream.

  func (c *cacheRepo) initStream(ctx context.Context, streamKey string) {
    if err := c.redis.XGroupCreateMkStream(ctx, streamKey, Envs.STREAM_CONSUMER_GROUP, "$").Err(); err != nil {
      if !strings.Contains(err.Error(), "BUSYGROUP") {
        Log.ErrorContext(ctx, "error creating stream group", "error", err)
      }
    }
  }

To write to the stream

 func (c *cacheRepo) writeToStream(ctx context.Context, streamKey, msg string) error {
    return c.redis.XAdd(ctx, &redis.XAddArgs{
      Stream:     streamKey, // provide the stream name
      // if the stream is not present, then it will give error,
      // to dynamically create the stream just like pubsub channel,
      // make this variable false
      NoMkStream: true, 
      // max length of message that will be present at any point of time in stream
      MaxLen:     int64(Envs.MAX_CHAT_LEN), 
      // default ID generator
      ID:         "*",
      Values: map[string]interface{}{
        "message": msg,
      },
    }).Err()
  }

Reading from the stream

...
  // writePump func
    streams, err := h.repo.redis.XRead(ctx, &redis.XReadArgs{
    Streams: []string{Envs.STREAM_KEY, lastMsgID},
    Block:   0,
  }).Result()

  if err != nil {
    Log.ErrorContext(ctx, "error reading from stream", "error", err)
    continue
  }

  for _, stream := range streams {
    for _, msg := range stream.Messages {
      lastMsgID = msg.ID
      payload := msg.Values["message"].(string)

      // if message writing takes more time than writeWait,
      // which can means client as slow internet
      // and then just hang the conn and UI will restablish it
      conn.SetWriteDeadline(time.Now().Add(writeWait))
      if err := conn.WriteMessage(websocket.TextMessage, []byte(payload)); err != nil {
        Log.ErrorContext(ctx, "error sending messsage to websocket", "error", err)
        return
      }
    }
  }

Since streams stores the messages, we can actually send chat history to the user

  func (c *cacheRepo) getMessagesFromStream(ctx context.Context, streamKey string) ([]string, error) {
    msgs, err := c.redis.XRevRangeN(ctx, streamKey, "+", "-", int64(Envs.MAX_CHAT_LEN)).Result()
    if err != nil {
      return nil, err
    }
    var result []string
    for _, msg := range msgs {
      result = append(result, msg.Values["message"].(string))
    }
    return result, nil
  }

Frontend code

In frontend, we use the native javascript websocket client to establish connection with the backend


  useEffect(() => {
    if (username) {
      const url = `${
        import.meta.env.VITE_API_URL
      }?username=${encodeURIComponent(username)}`;
      ws.current = new WebSocket(url);

      ws.current.onopen = () => {
        console.log("Connected to WebSocket server");
      };

      ws.current.onmessage = (event) => {
        const message: Message = JSON.parse(event.data);
        console.log(message);
        setMessages((prevMessages) => [...prevMessages, message]);
      };

      ws.current.onerror = (error) => {
        console.error("WebSocket error:", error);
      };

      ws.current.onclose = () => {
        console.log("Disconnected from WebSocket server");
      };

      return () => {
        ws.current?.close();
      };
    }
  }, [username]);

References

Some more learning resources which you can refer

Google OAuth 2.0 Flow in Golang and React.js

Aashish Koshti — Mon, 17 Feb 2025 12:54:25 +0000

In this blog I've shared my approach on integrating Google OAuth in my app (Golang for backend and React.js for frontend).

The code for this https://github.com/the-arcade-01/react-golang-oauth-flow

You can test the app itself here: https://oauth.arcade.build

Architecture

OAuth Flow

User requests for login from the client app
The client app hits login endpoint on the backend
The backend server generates the unqiue url of auth provider consent page and redirects the request
The consent page opens directly to the user requesting for granting permission
The user gives access to the permission, then auth provider calls the callback url of the backend server
The backend server then generates the access and refresh tokens and sets them in http only cookies
The backend server then redirects to the success page of the client app

Access & Refresh token Flow

Backend sets access and refresh token to http only cookie
Client calls protected routes using the access token
If the access token expires, the client then calls /refresh-token API using the refresh token from cookies
Backend then issues new access token and sets it in the cookie

Code overview

Here I won't put all the code from repo, I will just explain few code snippets so that you can implement it in your own way.

Backend
First, we need to create the oauth client with which we will interact with google auth

You can get the client ID and secret from here https://console.cloud.google.com/auth/clients

func newOauthConfig() *oauth2.Config {
    return &oauth2.Config{
        ClientID:     Envs.GOOGLE_CLIENT_ID,
        ClientSecret: Envs.GOOGLE_CLIENT_SECRET,
        RedirectURL:  Envs.AUTH_REDIRECT_URL, // http://localhost:8080/api/auth/callback
        Scopes:       []string{"https://www.googleapis.com/auth/userinfo.profile", "https://www.googleapis.com/auth/userinfo.email"},
        Endpoint:     google.Endpoint,
    }
}

Calling the auth provider for consent page

    url := h.config.OauthCfg.AuthCodeURL(state, oauth2.AccessTypeOffline) // AccessTypeOffline issues both access & refresh token
    http.Redirect(w, r, url, http.StatusTemporaryRedirect)

Handling callback function from auth provider

func (h *AuthHandlers) Callback(w http.ResponseWriter, r *http.Request) {
    code := r.URL.Query().Get("code")
    token, err := h.config.OauthCfg.Exchange(r.Context(), code)
    if err != nil {
        http.Redirect(w, r, fmt.Sprintf("%s=%s", config.Envs.APP_WEB_URL_LOGIN_ERROR, "exchange_failed"), http.StatusTemporaryRedirect)
        return
    }

    client := h.config.OauthCfg.Client(r.Context(), token)
    res, err := client.Get(config.Envs.GOOGLE_USER_INFO)
    if err != nil {
        http.Redirect(w, r, fmt.Sprintf("%s=%s", config.Envs.APP_WEB_URL_LOGIN_ERROR, "userinfo_failed"), http.StatusTemporaryRedirect)
        return
    }
    defer res.Body.Close()

    var userInfo map[string]string
    json.NewDecoder(res.Body).Decode(&userInfo)

    _, errResult := h.svc.Login(r.Context(), userInfo)
    if errResult != nil {
        http.Redirect(w, r, fmt.Sprintf("%s=%s", config.Envs.APP_WEB_URL_LOGIN_ERROR, "login_failed"), http.StatusTemporaryRedirect)
        return
    }

    h.setCookies(w, token)
    http.Redirect(w, r, config.Envs.APP_WEB_URL_LOGIN_SUCCESS, http.StatusTemporaryRedirect)
}

Setting the tokens in cookie

func (h *AuthHandlers) setCookies(w http.ResponseWriter, token *oauth2.Token) {
    http.SetCookie(w, &http.Cookie{
        Name:     "access_token",
        Value:    token.AccessToken,
        Expires:  token.Expiry,
        HttpOnly: config.Envs.HTTP_COOKIE_HTTPONLY, // true for production, false for local
        Secure:   config.Envs.HTTP_COOKIE_SECURE, // true for production, false for local
        SameSite: http.SameSiteLaxMode,
    })

    // NOTE: Refresh token is only issued at the first consent
    if token.RefreshToken != "" {
        http.SetCookie(w, &http.Cookie{
            Name:     "refresh_token",
            Value:    token.RefreshToken,
            Expires:  time.Now().Add(time.Duration(config.Envs.HTTP_REFRESH_TOKEN_EXPIRE) * time.Hour),
            HttpOnly: config.Envs.HTTP_COOKIE_HTTPONLY,
            Secure:   config.Envs.HTTP_COOKIE_SECURE,
            SameSite: http.SameSiteLaxMode,
        })
    }
}

To get the refresh token

// refreshToken is extracted from cookies
    tokenSource := h.config.OauthCfg.TokenSource(r.Context(), &oauth2.Token{
        RefreshToken: refreshToken.Value,
    })
    newToken, err := tokenSource.Token()

Frontend
Creating request interceptors using axios, interceptors are functions which are triggered before an api request or on its response or both.

Here on the response, we are checking if its 401 then we need to refresh the access token again.

We will use this api client for calling our apis.

const api = axios.create({
  baseURL: env.VITE_API_URL,
  withCredentials: true,
});

api.interceptors.request.use(
  (config) => {
    return config;
  },
  (error) => {
    return Promise.reject(error);
  }
);

api.interceptors.response.use(
  (response) => {
    return response;
  },
  async (error) => {
    const originalRequest = error.config;
    if (error.response.status === 401 && !originalRequest._retry) {
      originalRequest._retry = true;
      try {
        const res = await axios.post(
          `${env.VITE_API_URL}/auth/refresh-token`,
          {},
          { withCredentials: true }
        );
        if (res.status === 200) {
          return api(originalRequest);
        }
      } catch (refreshError) {
        return Promise.reject(refreshError);
      }
    }
    return Promise.reject(error);
  }
);

Creating a AuthProvider context

interface AuthContextType {
  isAuthenticated: boolean;
  user: User | null;
  login: () => void;
  logout: () => Promise<void>;
}

const AuthContext = createContext<AuthContextType | undefined>(undefined);

export const AuthProvider = ({ children }: { children: ReactNode }) => {
  const [isAuthenticated, setIsAuthenticated] = useState(false);
  const [user, setUser] = useState<User | null>(null);
  const [loading, setLoading] = useState(true);

  const checkAuthStatus = async () => {
    try {
      const response = await api.get("/auth/user/me");
      setUser(response.data.data);
      setIsAuthenticated(true);
    } catch (error) {
      setIsAuthenticated(false);
      setUser(null);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    checkAuthStatus();
  }, []);

  const login = () => {
    window.location.href = `${env.VITE_API_URL}/auth/login`;
  };

  const logout = async () => {
    await api.post("/auth/logout");
    setIsAuthenticated(false);
    setUser(null);
  };

  if (loading) {
    return <LoadingSpinner />;
  }

  return (
    <AuthContext.Provider value={{ isAuthenticated, user, login, logout }}>
      {children}
    </AuthContext.Provider>
  );
};

export const useAuth = () => {
  const ctx = useContext(AuthContext);
  if (ctx === undefined) {
    throw new Error("useAuth must be used within AuthProvider");
  }
  return ctx;
};

Protected Route component, using the context isAuthenticated bool to check user auth

const ProtectedRoute = ({ children }: ProtectedRouteProps) => {
  const { isAuthenticated } = useAuth();

  if (!isAuthenticated) {
    return <Navigate to="/login" replace />;
  }

  return children;
};

In App.tsx, making Profile page as protected route

<Route
  path="/profile"
  element={
    <ProtectedRoute>
      <Profile />
    </ProtectedRoute>
  }
/>

Future Improvements

Storing the refresh token in DB
- Since the refresh token is only returned on the first consent, its ideal to store it in DB as if it gets lost on the client app we can set it again after fetching it from the DB.
- Second, if in case we need to revoke the refresh token, we can delete it from DB, and revoke it from auth provider and then issue it again from login flow
- The refresh token has to be send on each login request (whether fetching it from DB if its not expired, or generating new one if its expired)

References:

Some more learning resources which you can refer

Docker setup for Go APIs

Aashish Koshti — Thu, 21 Dec 2023 11:18:35 +0000

In this blog post I've covered how to integrate Docker in your Backend Go APIs application.

What you will gain?

After reading this, you will be able to create your own DockerFile and docker-compose file and have a up and running application.

Code for this post can be found on my github
Code: https://github.com/the-arcade-01/go-api

PreRequisites

You've gone through the 1st and 2nd blog of this series as we will be using the same MySQL api project.
You should have Docker installed in your system.

Concepts

Previously, we have install MySQL on our machine and we were interacting with it on our machine, but now we'll leverage docker and run our MySQL database and our Go application in a container.

First lets create our docker config for our Go API app

DockerFile contains steps which creates the image, which you can think of like a snapshot of your code, which you run on a container, Container means an isolated environment in which your application runs.

DockerFile

# STEP 1
FROM golang:1.21.3

# STEP 2
WORKDIR /app

# STEP 3
COPY go.mod go.sum ./

# STEP 4
RUN go mod download

# STEP 5
COPY . .

# STEP 6
RUN go build -o ./bin/main main.go

# STEP 7
RUN chmod +x ./run.sh

# STEP 8
ENTRYPOINT [ "sh", "./run.sh" ]

Now, lets describe each steps of above file

FROM: in this step we download the base version of OS or dependency on which our application will run. Here, we are downloading golang:1.21.3 from DockerHub
WORKDIR: here we are specifying in which folder our application will be present on this above base machine.
COPY <src> <dest>: Now, we are copying our go.mod and go.sum file which contains all the dependencies.
RUN <cmd>: now, we are download all the dependencies which are required for the Go app.
In this step, we are copying all the files into our docker base machine.
Now, we will generate the binary executable for our Go app.
Now, this we're giving executable permission to our bash file run.sh which looks like this

run.sh

#!/bin/sh

/app/bin/main

this bash file contains the path of our binary executable, and when we will run our bash file, our binary executable will be called.
ENTRYPOINT: this tells which command to execute when we start our container. Here, run.sh will be called.

Now, we can setup docker-compose file

docker-compose is a single file which contains all the instructions of different services which are being used in the current application. This file helps us to run all the services in a single command.

docker-compose.yaml

version: "3.8"

services:
  db:
    container_name: "mysql_db"
    image: mysql:8.0.33
    networks:
      - default
    restart: always
    ports:
      - "3306:3306"
    environment:
      MYSQL_RANDOM_ROOT_PASSWORD: ${MYSQL_RANDOM_ROOT_PASSWORD}
      MYSQL_DATABASE: ${MYSQL_DATABASE}
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    volumes:
      - ./db.sql:/docker-entrypoint-initdb.d/0_init.sql
      - mysql_data:/var/lib/mysql
    command: --default-authentication-plugin=mysql_native_password
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      timeout: 20s
      retries: 10
  app:
    container_name: "go_api"
    build: .
    env_file:
      - .env
    ports:
      - "5000:5000"
    depends_on:
      db:
        condition: service_healthy

networks:
  default:

volumes:
  mysql_data:

Lets go through each keyword
services: inside this we list all our services which we are using, like database, redis, go app etc. Inside this you will see two config, first db config and second app config.

db config contains everything related to our MySQL database.

container_name: we assign a name to our container
image: here we specify which base version we want to use, here we using mysql:8.0.33 version.
networks: networks are use for establishing connection between multiple containers, here we want our Go app and db to connect to a same network, and are using default for that which will be created by docker itself when we run our file.
restart: if we encounter any error will starting the container, it will automatically restart untils its up.
ports: default mysql port 3306

NOTE: If you've MySQL currently running on your system, then you have to stop it or otherwise you will have connection error

Error starting userland proxy: listen tcp4 0.0.0.0:3306: bind: address already in use

On linux you can run the below command

sudo service mysql stop

environment: here we specify all the necessary env. variables as per MySQL docker docs
volumes: volumes are used for presisting data like eg. you ran a MySQL container and wants its data to be save even when you kill that container. Here we are specify two things
the below cmd will copy our db.sql file which contains our database and table schema queries, we are copying sql file in docker-entrypoint file. When we will run our MySQL container it will automatically run our db migrations.

- ./db.sql:/docker-entrypoint-initdb.d/0_init.sql

path at which our volume will be present

- mysql_data:/var/lib/mysql

command: specifying auth policy as mysql_native_password, as per docs its mentioned to use caching_sha2_password instead of this for version greater than 8.0, but this policy will also work fine.
healthcheck: this is an important feature, now we want our Go api app to start after the db has been initialized and is running, we specify a cmd which will get trigger and based on its success our apps which are depended on db will start. Here, the cmd we are using is a simple ping.

Now, lets see our app config
Here we specify the container name go_api, docker build path to be . which will pick the DockerFile, then we specify the env_file which looks like this
.env

PORT=:5000
DATABASE_URL=<user>:<password>@tcp(<mysql-container>:3306)/<database>?parseTime=true
DB_DRIVER=mysql
MYSQL_RANDOM_ROOT_PASSWORD=<password>
MYSQL_DATABASE=<database>
MYSQL_USER=<user>
MYSQL_PASSWORD=<password>

NOTE: In the DATABASE_URL, <mysql-container> value should be replaced with db, eg: DATABASE_URL=user:123@tcp(db:3306)/Practice?parseTime=true

depends_on: this tells on which other containers our current app is depending on, and if the other container is not initialized or is inactive, then our container will not start. Here we have specified

depends_on:
      db:
        condition: service_healthy

Running the Application

Now, we are ready to run our application
So, first we'll run

docker compose build

this will build an image of our application.

➜  go-api git:(main) docker compose build
[+] Building 35.2s (12/12) FINISHED                              docker:default
 => [app internal] load .dockerignore                                      0.1s
 => => transferring context: 2B                                            0.0s
 => [app internal] load build definition from Dockerfile                   0.2s
 => => transferring dockerfile: 218B                                       0.0s
 => [app internal] load metadata for docker.io/library/golang:1.21.3      10.3s
 => [app internal] load build context                                      0.4s
 => => transferring context: 13.87kB                                       0.3s
 => [app 1/7] FROM docker.io/library/golang:1.21.3@sha256:b113af1e8b06f06  0.0s
 => CACHED [app 2/7] WORKDIR /app                                          0.0s
 => [app 3/7] COPY go.mod go.sum ./                                        0.5s
 => [app 4/7] RUN go mod download                                          3.5s
 => [app 5/7] COPY . .                                                     0.9s
 => [app 6/7] RUN go build -o ./bin/main main.go                          17.1s
 => [app 7/7] RUN chmod +x ./run.sh                                        1.1s
 => [app] exporting to image                                               0.7s
 => => exporting layers                                                    0.7s
 => => writing image sha256:bf43ed8cbf2418a6f75f9b4395a630557a2c8ee85db65  0.0s
 => => naming to docker.io/library/go-api-app                              0.1s

Now, after the image has been build, we can run the below cmd to start our application

docker compose up

this will start our application and our MySQL container

➜  go-api git:(main) docker compose up              
[+] Running 2/2
 ✔ Container mysql_db  Created                                             0.0s 
 ✔ Container go_api    Recreated                                           0.4s 
Attaching to go_api, mysql_db

and

mysql_db  | 2023-12-21T10:35:29.836732Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.33'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
go_api    | Connected to DB
go_api    | server running on port:5000

Testing

Now, we can test the todo apis

➜  go-api git:(main) curl -X POST 'http://localhost:5000/todos' -d '{"task": "Learn Docker with Go", "completed": false}'
Todo added!!%

fetch todos

➜  go-api git:(main) curl -X GET 'http://localhost:5000/todos' 
[{"id":1,"task":"Learn Docker with Go","completed":false,"created_at":"2023-12-21T10:52:41Z","updated_at":"2023-12-21T10:52:41Z"}]

Logs will look like this

go_api    | Connected to DB
go_api    | server running on port:5000
go_api    | 2023/12/21 10:52:22 "GET http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:41316 - 200 119B in 18.831971ms
go_api    | 2023/12/21 10:52:41 "POST http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:45632 - 200 12B in 29.13446ms
go_api    | 2023/12/21 10:53:10 "GET http://localhost:5000/todos HTTP/1.1" from 172.20.0.1:57434 - 200 248B in 643.34µs

Extras

You can even check which all containers are running, using below cmd

➜  go-api git:(main) docker ps        
CONTAINER ID   IMAGE              COMMAND                  CREATED         STATUS                   PORTS                                                  NAMES
a8a9f20533cd   go-api-app         "sh ./run.sh"            2 minutes ago   Up 2 minutes             0.0.0.0:5000->5000/tcp, :::5000->5000/tcp              go_api
f6d7a0153ac0   mysql:8.0.33       "docker-entrypoint.s…"   3 days ago      Up 2 minutes (healthy)   0.0.0.0:3306->3306/tcp, :::3306->3306/tcp, 33060/tcp   mysql_db

and you can also ssh into our MySQL container using its container id

➜  go-api git:(main) docker exec -it f6d7a0153ac0 sh            
sh-4.4#

Now, we will check in our mysql table whether the data is present or not. Use the same user and password which you defined in .env file.

sh-4.4# mysql -u user -p 
Enter password:

Now, check data in table Todos

mysql> use Practice;
Database changed
mysql> show tables;
+--------------------+
| Tables_in_Practice |
+--------------------+
| Todos              |
+--------------------+
1 row in set (0.01 sec)
mysql> select * from Todos;
+----+----------------------+-----------+---------------------+---------------------+
| id | task                 | completed | created_at          | updated_at          |
+----+----------------------+-----------+---------------------+---------------------+
|  1 | Learn Docker with Go |         0 | 2023-12-21 10:52:41 | 2023-12-21 10:52:41 |
+----+----------------------+-----------+---------------------+---------------------+
1 rows in set (0.00 sec)

Conclusion

That's it, we covered a lot of stuff in this blog, I encourage you to read more about Docker and create your own personal project using it.

Github: https://github.com/the-arcade-01/go-api

Thanks for reading till the end, really appreciate it!!

User Authentication in Go APIs: JWT, Password Hashing, and MySQL

Aashish Koshti — Sun, 10 Sep 2023 20:00:34 +0000

In this blog post, you'll learn how to build user authentication in Go APIs. We'll create user, store there password securely using bcrypt in MySQL database and create protected routes by using JWT auth tokens using go-chi.

PreRequisites

Basic of Go
Basic of MySQL
Basic understanding of JWT Tokens
Should have gone through the previous blog posts for basic understanding
- Write APIs in Go using go-chi
- Storing and retrieving data from MySQL in Go APIs

Code

Full code of this blog can be found in my Github repo the-arcade-01/go-auth-api

So, the first step is to create the database with the User table.

For the sake of simplicity, we are only storing 3 fields for now.

Password entered by user will be hashed using bcrypt and will be stored in hash field.

CREATE DATABASE IF NOT EXISTS `Practice`;

CREATE TABLE IF NOT EXISTS `Practice`.`User` (
    `id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    `email` VARCHAR(255) NOT NULL,
    `hash` VARCHAR(255) NOT NULL
);

Now create an environment file, add your MySQl <user>, <password> and <database> into it.
.env

PORT=:5000
DATABASE_URL=<user>:<password>@tcp(localhost)/<database>?parseTime=true
DB_DRIVER=mysql
JWT_SECRET_KEY=<secret_key>

Now, this been done we will install some external libraries that we will be using in this project

go get -u github.com/go-chi/chi/v5        // library for creating APIs in Go
go get golang.org/x/crypto/bcrypt         // for hashing password
go get github.com/joho/godotenv           // loading .env variables
go get github.com/go-sql-driver/mysql     // DB driver for MySQL
go get github.com/go-chi/jwtauth/v5       // JWT library

main.go

package main

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"
    "strconv"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
    "github.com/go-chi/jwtauth/v5"
    "github.com/joho/godotenv"
    "golang.org/x/crypto/bcrypt"

    _ "github.com/go-sql-driver/mysql"
)

var (
    DATABASE_URL, DB_DRIVER, JWT_SECRET_KEY, PORT string
)

/*
 *  Loads enviroment variables for .env
*/
func init() {
    err := godotenv.Load()
    if err != nil {
        log.Fatalln("Coudn't load env file!!")
    }

    DATABASE_URL = os.Getenv("DATABASE_URL")
    DB_DRIVER = os.Getenv("DB_DRIVER")
    PORT = os.Getenv("PORT")
    JWT_SECRET_KEY = os.Getenv("JWT_SECRET_KEY")
}

/*
 *  DB connection is created in this function
*/
func DBClient() (*sql.DB, error) {
    db, err := sql.Open(DB_DRIVER, DATABASE_URL)
    if err != nil {
        return nil, err
    }

    if err := db.Ping(); err != nil {
        return nil, err
    }
    fmt.Println("Connected to DB!!")
    return db, nil
}

/*
 *  This funtion returns the JWT auth structure
 *  We passed our own JWT_SECRET_KEY to generate it
*/
func GenerateAuthToken() *jwtauth.JWTAuth {
    tokenAuth := jwtauth.New("HS256", []byte(JWT_SECRET_KEY), nil)
    return tokenAuth
}

type Server struct {
    Router    *chi.Mux
    DB        *sql.DB
    AuthToken *jwtauth.JWTAuth
}

func CreateServer(db *sql.DB) *Server {
    server := &Server{
        Router:    chi.NewRouter(),
        DB:        db,
        AuthToken: GenerateAuthToken(),
    }
    return server
}

/*
 *  go-chi's middleware library provides us with different inbuilt functionality
 *  like Clean path, logger, cors etc. read here (https://go-chi.io/#/pages/middleware)
 * Here we're using Logger which will log inbound requests
*/
func (server *Server) MountMiddleware() {
    server.Router.Use(middleware.Logger)
}

/*
 * Base url: http:localhost:5000/user
*/
func (server *Server) MountHandlers() {
    server.Router.Route("/user", func(userRouter chi.Router) {

    /*  These endpoins will be accessible to user without auth
     *  POST /user/login    =   login user
     *  POST /user          =   create user
    */
        userRouter.Post("/login", server.LoginUser)
        userRouter.Post("/", server.CreateUser)

        userRouter.Group(func(r chi.Router) {
      /*  We mount the Verifier and Authenticator to our request
       *  All APIs inside this group will require JWT Token in Auth Headers
      */
            r.Use(jwtauth.Verifier(server.AuthToken))
            r.Use(jwtauth.Authenticator)

      /*  GET /user/{id}    =   get user with specific ID
      */
            r.Get("/{id}", server.GetUser)
        })
    })
}

func main() {
    db, err := DBClient()
    if err != nil {
        log.Fatal(err)
    }

    server := CreateServer(db)
    server.MountMiddleware()
    server.MountHandlers()
    fmt.Printf("server running on port%v\n", PORT)
    http.ListenAndServe(PORT, server.Router)
}

Now, we'll create neccessary schemas that we'll use in this project.

type User struct {
    Id    int    `json:"id"`
    Email string `json:"email"`
    Hash  string `json:"hash"`
}

type UserRequestBody struct {
    Email    string `json:"email"`
    Password string `json:"password"`
}

type Response struct {
    Id int `json:"id"`
}

func ScanRow(rows *sql.Rows) (*User, error) {
    user := new(User)
    err := rows.Scan(&user.Id, &user.Email, &user.Hash)
    if err != nil {
        return nil, err
    }
    return user, nil
}

Now, the actual implementation and the most important parts, the routes for creating user, login user and get users details.

Lets start with create user route first.

/*
 * We pass user's password into this function and
 * call bcrypts function to give us the hashed password
*/
func getHashPassword(password string) (string, error) {
    bytePassword := []byte(password)
    hash, err := bcrypt.GenerateFromPassword(bytePassword, bcrypt.DefaultCost)
    if err != nil {
        return "", err
    }
    return string(hash), nil
}

/*
 * We parse the user request body, takes the email, password
 * hashes the password and then runs insert query to insert in DB
 * on success returns the Id of the record
*/
func (server *Server) CreateUser(w http.ResponseWriter, r *http.Request) {
    userReqBody := new(UserRequestBody)
    if err := json.NewDecoder(r.Body).Decode(userReqBody); err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please provide the correct input!!"))
        return
    }

    hashPassword, err := getHashPassword(userReqBody.Password)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("Something bad happened on the server :("))
        return
    }

    query := `INSERT INTO User (email, hash) VALUES (?, ?)`
    result, err := server.DB.Exec(query, userReqBody.Email, hashPassword)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("Something bad happened on the server :("))
        return
    }
    recordId, _ := result.LastInsertId()
    response := Response{
        Id: int(recordId),
    }

    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(response)
}

Now, we'll write the Login user endpoint

/*
 * We compare the hashed password in DB with the password entered by user
*/
func checkPassword(hashPassword, password string) bool {
    err := bcrypt.CompareHashAndPassword([]byte(hashPassword), []byte(password))
    return err == nil
}

/*
 * We parse the request body, fetch user details based on email entered,
 * check the hashed password with user's entered password
 * generate claims and create token and return it in response
*/
func (server *Server) LoginUser(w http.ResponseWriter, r *http.Request) {
    userReqBody := new(UserRequestBody)
    if err := json.NewDecoder(r.Body).Decode(userReqBody); err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please provide the correct input!!"))
        return
    }
    query := `SELECT * FROM User where email = ?`
    rows, err := server.DB.Query(query, userReqBody.Email)
    if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please provide the correct input!!"))
        return
    }
    var user *User
    for rows.Next() {
        user, err = ScanRow(rows)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte("Something bad happened on the server :("))
            return
        }
    }

    if !checkPassword(user.Hash, userReqBody.Password) {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Incorrect password please check again"))
        return
    }

  /* After verify password, we want to generate user specific token
   * for that we mask few of the user details in the jwt token
   * these are called as claims and are used for verifying user w.r.t token
  */
    claims := map[string]interface{}{"id": user.Id, "email": user.Email}
    _, tokenString, err := server.AuthToken.Encode(claims)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("Something bad happened on the server :("))
        return
    }

    w.WriteHeader(http.StatusOK)
    w.Write([]byte(tokenString))
}

Now, lets write the final endpoint get user by his id

/* This GET /user/{id} will require the JWT token
 * generated from POST /user/login in auth headers
*/
func (server *Server) GetUser(w http.ResponseWriter, r *http.Request) {

  // We get the 'id' from URL parameters of the request
  id := chi.URLParam(r, "id")
    userId, err := strconv.Atoi(id)

  if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please provide the correct input!!"))
        return
    }

  /* After the Verifier and Authenticator have successful validated this request
   * We destructure the claims from the request and get the userId from claims
   * We then check whether the userId from claims is same as the userId for which
   * the request has been hit (from url params), if not that means user is using
   * different JWT token and hence unauthorized.
  */
    _, claims, _ := jwtauth.FromContext(r.Context())
    userIdFromClaims := int(claims["id"].(float64))

    if userId != userIdFromClaims {
        w.WriteHeader(http.StatusUnauthorized)
        w.Write([]byte("You're not authorized >("))
        return
    }

    query := `SELECT * FROM User WHERE id = ?`

    rows, err := server.DB.Query(query, userId)
    if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please provide the correct input!!"))
        return
    }

    var user *User
    for rows.Next() {
        user, err = ScanRow(rows)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte("Something bad happened on the server :("))
            return
        }
    }
    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(user)
}

Now, thats been done, we can finally hit our endpoints and see the response

➜  go-auth-api git:(main) ✗ curl -X POST 'http://localhost:5000/user' -d '{"email": "test@test.com", "password": "test"}'

{"id":1}

➜  go-auth-api git:(main) ✗ curl -X POST 'http://localhost:5000/user/login' -d '{"email": "test@test.com", "password": "test"}'

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InRlc3RAdGVzdC5jb20iLCJpZCI6Mn0.M3MPy4rxJVDn_lxSnIXpfZavWC-gPr3325WnLT26NHw

➜  go-auth-api git:(main) ✗ curl -X GET 'http://localhost:5000/user/2' -H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InRlc3RAdGVzdC5jb20iLCJpZCI6Mn0.M3MPy4rxJVDn_lxSnIXpfZavWC-gPr3325WnLT26NHw'

{"id":1,"email":"test@test.com","hash":"$2a$10$P7.mUSvVrjx3N6mPEAD19.N9RoGEtGXNuz.XfGzeKmB9fsFqaZUbq"}

Logs will look like this as we used go-chi middleware Logger

➜  go-auth-api git:(main) ✗ make run
Connected to DB!!
server running on port:5000
2023/09/10 01:05:28 "POST http://localhost:5000/user HTTP/1.1" from 127.0.0.1:43244 - 200 9B in 146.277249ms
2023/09/10 01:06:55 "POST http://localhost:5000/user/login HTTP/1.1" from 127.0.0.1:49428 - 200 124B in 78.764985ms
2023/09/10 01:10:18 "GET http://localhost:5000/user/2 HTTP/1.1" from 127.0.0.1:42164 - 200 103B in 705.163µs

I encourage you to create another user and check whether he is able to get different users details with his own token.

Same can be seen inside your MySQL db.

mysql> select * from User;
+----+---------------+--------------------------------------------------------------+
| id | email         | hash                                                         |
+----+---------------+--------------------------------------------------------------+
|  1 | test@test.com | $2a$10$P7.mUSvVrjx3N6mPEAD19.N9RoGEtGXNuz.XfGzeKmB9fsFqaZUbq |
+----+---------------+--------------------------------------------------------------+
1 row in set (0.00 sec)

Conclusion

That's it, we covered a lot of stuff in this blog, I encourage you to write code own your own and explore different implementations as well.

Github: https://github.com/the-arcade-01/go-auth-api

Thanks for reading till the end, really appreciate it!!

Storing and retrieving data from MySQL in Go APIs

Aashish Koshti — Sun, 03 Sep 2023 10:00:47 +0000

In the last blog post, we create basic CRUD endpoints for a Todos app. Now, we'll use a real database like MySQL to store and reterive data for its tables.

PreRequisites

Should have gone through the previous blog post for basic understanding Write APIs in Go using go-chi.

MySQL install in your system and basic working knowledge.

Code

So, lets first setup our tables that we will use for storing the Todos. First, we'll create a database Practice, then inside that database we'll create Todos table.

CREATE DATABASE IF NOT EXISTS `Practice`;

CREATE TABLE IF NOT EXISTS `Practice`.`Todos` (
    `id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    `task` VARCHAR(255) NOT NULL,
    `completed` INT NOT NULL DEFAULT 0,
    `created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
    `updated_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

Note that we've added new fields in this Todos tables, id, created_at, updated_at. We'll look how to use these when we perform operations on this table.

Now create an environment file, add your MySQl <user>, <password> and <database> into it.
.env

PORT=:5000
DATABASE_URL=<user>:<password>@tcp(localhost)/<database>?parseTime=true
DB_DRIVER=mysql

Now, lets install few dependencies

go get github.com/joho/godotenv
go get github.com/go-sql-driver/mysql

Now lets start writing code for connecting to database

main.go

import (
    "database/sql"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"

    "github.com/go-chi/chi/v5"
    _ "github.com/go-sql-driver/mysql"
    "github.com/joho/godotenv"
)

var (
    DATABASE_URL, DB_DRIVER, PORT string
)

func init() {
    err := godotenv.Load()
    if err != nil {
        log.Fatalln("Couldn't load env on startup!!")
    }
    DATABASE_URL = os.Getenv("DATABASE_URL")
    DB_DRIVER = os.Getenv("DB_DRIVER")
    PORT = os.Getenv("PORT")
}

func DBClient() (*sql.DB, error) {
    db, err := sql.Open(DB_DRIVER, DATABASE_URL)
    if err != nil {
        return nil, err
    }
    if err := db.Ping(); err != nil {
        return nil, err
    }
    fmt.Println("Connected to DB")
    return db, nil
}

Here, we first loaded our environment variables in init function, then we created a DBClient function in which we used Go's inbuilt library database/sql for creating a connection with the database, this function returns a client using which we interact/perform operations on tables.

Now, lets make changes to our Server struct, so that it holds the DB client.

type Server struct {
    Router *chi.Mux
    DB     *sql.DB
}

func CreateServer(db *sql.DB) *Server {
    server := &Server{
        Router: chi.NewRouter(),
        DB:     db,
    }
    return server
}

func main() {
    db, err := DBClient()
    if err != nil {
        log.Fatalln("Couldn't connect to DB")
    }
    server := CreateServer(db)
    server.MountHandlers()

    fmt.Println("server running on port:5000")
    http.ListenAndServe(PORT, server.Router)
}

Now with these changes you can start and check whether you are able to connect to the database or not.

➜  go-api git:(main) ✗ make run
Connected to DB
server running on port:5000

NOTE: If you're getting any errors check whether you've imported all the necessary libraries and create credentials are present in .env file

Now, lets write the GetTodos and AddTodo function again with the use of DB client
Since, we've our DB client present inside Server struct, so to access it in our GetTodos and AddTodo function, we have to make them as methods of Server struct and we also have to make changes in MountHandlers function.

func (server *Server) MountHandlers() {
    server.Router.Get("/greet", Greet)

    todosRouter := chi.NewRouter()
    todosRouter.Group(func(r chi.Router) {
        // make these as a methods of Server struct as to access DB client
        r.Get("/", server.GetTodos)
        r.Post("/", server.AddTodo)
    })

    server.Router.Mount("/todos", todosRouter)
}

type Todo struct {
    Id        int       `json:"id"`
    Task      string    `json:"task"`
    Completed bool      `json:"completed"`
    CreatedAt time.Time `json:"created_at"`
    UpdatedAt time.Time `json:"updated_at"`
}

type TodoRequestBody struct {
    Task      string `json:"task"`
    Completed bool   `json:"completed"`
}

func scanRow(rows *sql.Rows) (*Todo, error) {
    todo := new(Todo)
    err := rows.Scan(&todo.Id,
        &todo.Task,
        &todo.Completed,
        &todo.CreatedAt,
        &todo.UpdatedAt,
    )
    if err != nil {
        return nil, err
    }
    return todo, nil
}

func (server *Server) AddTodo(w http.ResponseWriter, r *http.Request) {
    todo := new(TodoRequestBody)
    if err := json.NewDecoder(r.Body).Decode(todo); err != nil {
        w.WriteHeader(http.StatusBadRequest)
        w.Write([]byte("Please enter a correct Todo!!"))
        return
    }

    query := `INSERT INTO Todos (task, completed) VALUES (?, ?)`
    _, err := server.DB.Exec(query, todo.Task, todo.Completed)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("Something bad happened on the server :("))
        return
    }

    w.WriteHeader(http.StatusOK)
    w.Write([]byte("Todo added!!"))
}

func (server *Server) GetTodos(w http.ResponseWriter, r *http.Request) {
    query := `SELECT * FROM Todos ORDER BY created_at DESC`

    rows, err := server.DB.Query(query)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        w.Write([]byte("Something bad happened on the server :("))
        return
    }

    var todos []*Todo

    for rows.Next() {
        todo, err := scanRow(rows)
        if err != nil {
            w.WriteHeader(http.StatusInternalServerError)
            w.Write([]byte("Something bad happened on the server :("))
            return
        }
        todos = append(todos, todo)
    }

    w.WriteHeader(http.StatusOK)
    json.NewEncoder(w).Encode(todos)
}

In GetTodos function, we created the query to fetch all the Todos from Todos table sorted in descending order based on created at. DB client provides us with a method Query with which we can run our query, its returns rows, we then destructure them and returns todos array as json.

In AddTodo function, we first take the request body and map it into a TodoRequestBody and then create a insert query, we execute this query using Exec method provided by DB client.

Hit the below endpoints using curl

➜  go-api git:(main) ✗ curl -X POST 'http://localhost:5000/todos' -d '{"task": "Learn Go", "completed": false}'
Todo added!!

➜  go-api git:(main) ✗ curl -X GET 'http://localhost:5000/todos'
[{"id":1,"task":"Learn Go","completed":false,"created_at":"2023-09-03T15:18:54Z","updated_at":"2023-09-03T15:18:54Z"}]

You can also see the records directly into your MySQL table as well

mysql> SELECT * FROM Todos ORDER BY created_at DESC;
+----+----------+-----------+---------------------+---------------------+
| id | task     | completed | created_at          | updated_at          |
+----+----------+-----------+---------------------+---------------------+
|  1 | Learn Go |         0 | 2023-09-03 15:18:54 | 2023-09-03 15:18:54 |
+----+----------+-----------+---------------------+---------------------+
1 row in set (0.01 sec)

I encourage you to write PUT /todos, DELETE /todos endpoint on your own.

Conclusion

That's it, in this blog post we covered how we can connect and use MySQL databases and perform operations on tables and return response with our APIs.

Github: https://github.com/the-arcade-01/go-api

Thanks for reading till the end, really appreciate it!!

DEV Community: Aashish Koshti

Launched my iOS app - Tuneo Offline Music Player

Building With Docker: Images, Containers, Multi-Stage builds & Optimization

Prerequisites.

Understanding Image Layers & Caching.

Writing a DockerFile.

Building Image & Running Containers.

Multi-Stage builds.

Optimizing Images & Best Practices.

Summary.

Conclusion.

Docker Basics: Beginner's Introduction

History of Developing & Deploying application.

Docker Architecture.

Image, Containers & Registry.

Some CLI CMDs & Practical Example.

Summary.

Conclusion.

AWS Cognito Authentication with Golang

Pre-Requisites

AWS Cognito User Pool Setup

Code Explanation

API Testing

Conclusion

References

Golang LLD: Design a Task Scheduler

Requirements

Thought Process

Code structure

Code overview

Conclusion

Understanding Authentication & Authorization (JWT, OAuth2.0, Session) Concepts

What is Authentication & Authorization?

Auth Strategies

Basic Authentication

JWT Authentication

OAuth 2.0 Authorization

Session-Based Authentication

Conclusion

References

Golang LLD: Design a Cache System (LRU, LFU, FIFO)

Requirements

Code structure

Code explanation

Further improvements

Conclusion

Using Redis to build a Distributed Chat App in Go & React.js

Theory

First of all, what does Distributed even means?

Now, let's talk about Websockets

Redis & How we are using Streams and PubSub for our chat.

Redis PubSub

Redis Streams

Code Overview

Backend code

Frontend code

References

Google OAuth 2.0 Flow in Golang and React.js

Architecture

Code overview

Future Improvements

References:

Docker setup for Go APIs

What you will gain?

PreRequisites

Concepts

Running the Application

Testing

Extras

Conclusion

User Authentication in Go APIs: JWT, Password Hashing, and MySQL

PreRequisites

Code

Conclusion

Storing and retrieving data from MySQL in Go APIs

PreRequisites

Code

Conclusion

First of all, what does `Distributed` even means?

Now, let's talk about `Websockets`

`Redis` & How we are using `Streams` and `PubSub` for our chat.

Redis `PubSub`

Redis `Streams`