DEV Community: Vishal Shukla

Intermediate Go Tutorial: Building Your Own Redis

Vishal Shukla — Tue, 01 Apr 2025 10:58:03 +0000

This tutorial will guide you through building your own Redis-like in-memory cache system using Go. By the end of this tutorial, you'll have a functional cache that supports basic operations such as setting and getting keys, LRU eviction, and persistence to disk.

Prerequisites

Before you get started, make sure you have the following prerequisites:

Basic Knowledge of Go: Familiarity with Go programming, including understanding of data types, structs, control flow, and basic concurrency (goroutines and channels).
Understanding of Caching and LRU Algorithm: Knowing how caches work and what the LRU (Least Recently Used) eviction strategy is will help you understand the logic behind cache management.

Once you have these prerequisites in place, you're ready to start!

Folder Structure

redis-clone/                  
│
├── store/                      
│   ├── cache.go               
│   └── persistence.go          
│
├── cache.aof                   
├── main.go                     
├── go.mod                      
├── go.sum                      
└── README.md

Implementation

`cache.go`: Implementing the LRU Cache

In the store/ folder, the cache.go file contains the core logic for our in-memory LRU (Least Recently Used) cache. The LRU cache is designed to store key-value pairs and automatically evict the least recently used entries when the cache reaches its maximum size.

package store

import (
    "container/list"
    "fmt"
    "log"
    "sync"
    "time"
)

/*
Item represents a single cache entry in the LRU (Least Recently Used) cache.
It contains the value of the cached item, the expiration time (if any),
and a reference to its corresponding list element in the LRU doubly linked list.
The reference to the list element allows the cache to efficiently update the position of the item
when it is accessed, ensuring the correct eviction of the least recently used items.
*/
type Item struct {
    value     string        // The value stored in the cache item
    expiresAt time.Time     // The expiration time for the cache item (if applicable)
    elem      *list.Element // A reference to the element in the LRU doubly linked list
}

/*
Cache represents an LRU (Least Recently Used) cache that stores key-value pairs.
The cache supports setting and getting items, with optional TTL (Time To Live) for each cache entry.
When the cache exceeds its maximum size, it evicts the least recently used item to make room for new entries.
Additionally, the cache can persist commands to an AOF (Append-Only File) for persistence between restarts.
*/
type Cache struct {
    mu      sync.RWMutex     // Mutex to ensure thread safety for concurrent access to the cache
    items   map[string]*Item // Map that stores cache items by their key
    lru     *list.List       // Doubly linked list used for tracking the access order of cache items (for LRU eviction)
    maxSize int              // Maximum number of items the cache can hold before eviction occurs
    persist *Persistence     // Optional persistence mechanism for appending commands to a file
}

/*
NewCache initializes and returns a new Cache instance with the specified max size.
The cache starts with an empty linked list and an empty item map, and optionally supports persistence.
*/
func NewCache(maxSize int, persist *Persistence) *Cache {
    return &Cache{
        items:   make(map[string]*Item), // Initialize the map to store cache items
        lru:     list.New(),             // Initialize the doubly linked list to track the LRU order
        maxSize: maxSize,                // Set the maximum size for the cache
        persist: persist,                // Optionally set the persistence mechanism
    }
}

/*
Set stores a key-value pair in the cache, optionally with a TTL (Time To Live).
If the cache exceeds the maximum size, the least recently used (LRU) item is evicted.
If persistence is enabled, the SET command is appended to the AOF file.
*/
func (c *Cache) Set(key string, value string, ttl time.Duration, replaying bool) {
    c.mu.Lock()         // Lock the cache to ensure thread-safe access
    defer c.mu.Unlock() // Unlock once the operation is complete

    var expiresAt time.Time
    if ttl != 0 {
        expiresAt = time.Now().Add(ttl) // Set the expiration time if TTL is provided
    }

    // Check if the key already exists in the cache
    if item, exists := c.items[key]; exists {
        // Update the value and expiration time of the existing item
        item.value = value
        item.expiresAt = expiresAt
        // Move the item to the front of the LRU list to mark it as recently used
        c.lru.MoveToFront(item.elem)
    } else {
        // Create a new item and insert it at the front of the LRU list
        elem := c.lru.PushFront(key)
        c.items[key] = &Item{
            value:     value,
            expiresAt: expiresAt,
            elem:      elem, // Store the list element reference with the item
        }
    }

    // Evict the least recently used item if the cache exceeds the max size
    if c.lru.Len() > c.maxSize {
        oldest := c.lru.Back() // Get the least recently used item (the oldest in the list)
        if oldest != nil {
            delete(c.items, oldest.Value.(string)) // Remove the item from the map
            c.lru.Remove(oldest)                   // Remove the item from the LRU list
        }
    }

    // Persist the command in the AOF file if persistence is enabled and not replaying
    if c.persist != nil && !replaying {
        cmd := fmt.Sprintf("SET %s %s", key, value)
        err := c.persist.Append(cmd)
        if err != nil {
            log.Println("Failed to append to AOF file:", err)
        }
    }
}

/*
Get retrieves the value associated with a key from the cache.
If the key exists and has not expired, it is moved to the front of the LRU list.
If the key does not exist or has expired, it is removed from the cache and not returned.
*/
func (c *Cache) Get(key string) (string, bool) {
    c.mu.Lock()         // Lock the cache to ensure thread-safe access
    defer c.mu.Unlock() // Unlock once the operation is complete

    // Check if the key exists in the cache
    item, exists := c.items[key]
    if !exists {
        return "", false // Return false if the key doesn't exist
    }

    // Check if the item has expired
    if !item.expiresAt.IsZero() && time.Now().After(item.expiresAt) {
        // If expired, remove it from the LRU list and delete it from the cache
        c.lru.Remove(item.elem)
        delete(c.items, key)
        return "", false
    }

    // Move the item to the front of the LRU list to mark it as recently used
    c.lru.MoveToFront(item.elem)

    // Return the value of the item along with a true flag indicating success
    return item.value, true
}

/*
StartCleaningServer periodically runs a cleanup task that removes expired items from the cache.
This task is triggered every minute and ensures the cache does not hold stale data.
*/
func (c *Cache) StartCleaningServer() {
    ticker := time.NewTicker(time.Minute * 1) // Set the cleanup interval to 1 minute
    defer ticker.Stop()

    // Run the cleanup process at regular intervals
    for range ticker.C {
        log.Println("Running cache cleanup")
        c.cleanExpiredItems() // Perform the cleanup of expired items
    }
}

/*
cleanExpiredItems iterates through the cache and removes any expired items.
It checks each item in the LRU list, and if the item has expired, it is removed from both the list and the cache.
Additionally, if the cache exceeds its maximum size, the least recently used item is evicted.
*/
func (c *Cache) cleanExpiredItems() {
    c.mu.Lock()         // Lock the cache for thread-safe access
    defer c.mu.Unlock() // Unlock once the operation is complete

    // Iterate through the LRU list from the back (oldest) to the front (most recently used)
    for e := c.lru.Back(); e != nil; e = e.Prev() {
        item := c.items[e.Value.(string)]

        // Skip items that do not have an expiration time
        if item.expiresAt.IsZero() {
            continue
        }

        // Remove expired items
        if time.Now().After(item.expiresAt) {
            c.lru.Remove(e)                   // Remove the expired item from the LRU list
            delete(c.items, e.Value.(string)) // Remove the expired item from the cache map
            log.Printf("Removed expired item: %s\n", e.Value)
        }
    }

    // If the cache exceeds the maximum size, evict the least recently used item
    if c.lru.Len() > c.maxSize {
        oldest := c.lru.Back()
        if oldest != nil {
            delete(c.items, oldest.Value.(string))
            c.lru.Remove(oldest)
        }
    }
}

`persistence.go`: Managing Persistence for the Cache

In the store/ folder, the persistence.go file is responsible for handling the persistence logic of our custom Redis-like in-memory cache. Specifically, it focuses on storing commands to a file, ensuring that we can recover the cache's state after a restart. This is done using the Append-Only File (AOF) strategy, which logs every write operation to disk.

package store

import (
    "os"
)

/*
*
Persistence represents the mechanism for persisting the cache commands to a file.
It uses an append-only file (AOF) to persist commands that modify the cache state.
*/
type Persistence struct {
    file *os.File // The file used to store the commands
}

/*
*
NewAOF initializes and returns a new Persistence instance for appending commands to a file.
It opens the specified file in append mode, creating it if it doesn't exist.
*/
func NewAOF(filename string) (*Persistence, error) {
    file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        return nil, err
    }

    return &Persistence{file: file}, nil
}

/*
*
Append writes a command to the AOF file.
The command is written as a string followed by a newline character.
*/
func (p *Persistence) Append(cmd string) error {
    _, err := p.file.WriteString(cmd + "\n")
    return err
}

/*
*
Close closes the AOF file when the server is shutting down.
*/
func (p *Persistence) Close() {
    p.file.Close()
}

`main.go`: Starting the Redis-like Server

The main.go file serves as the entry point to our custom Redis-like server. It contains the initialization code for setting up and running the server, which includes loading the cache, setting up persistence, and listening for client connections. This file ties together all the components of the server and gets everything running.

package main

import (
    "bufio"
    "fmt"
    "log"
    "net"
    "os"
    "strings"
    "time"

    "github.com/vishaaxl/redis-clone/store"
)

/*
Server represents a simple Redis-like server that handles connections and processes Redis commands such as SET and GET.
The server uses an in-memory cache (LRU cache) to store key-value pairs, which is backed by an Append-Only File (AOF) persistence mechanism.
The server is capable of replaying the AOF file during startup to restore the cache state and handles incoming client connections to execute Redis-like commands.
*/
type Server struct {
    cache *store.Cache // Cache used to store key-value pairs
}

/*
NewServer initializes a new instance of the Server.
It creates a Cache instance with AOF persistence enabled, reading from the specified AOF file.
It also replays the AOF file to restore the cache state and starts a background cleaning process for the cache.
*/
func NewServer(aofFilename string) (*Server, error) {
    // Initialize AOF persistence
    persist, err := store.NewAOF(aofFilename)
    if err != nil {
        return nil, err
    }

    // Create the cache with a maximum size of 5 items and enable AOF persistence
    cache := store.NewCache(5, persist)

    // Replay the AOF file to restore the cache state
    if err := replayAOF(aofFilename, cache); err != nil {
        return nil, err
    }

    // Start the cache cleaning server in the background
    go cache.StartCleaningServer()

    return &Server{
        cache: cache, // Initialize the server with the cache
    }, nil
}

/*
replayAOF reads the AOF file and replays the commands to restore the cache state.
It processes each "SET" command in the AOF file and sets the corresponding key-value pair in the cache.
*/
func replayAOF(aofFilename string, cache *store.Cache) error {
    // Open the AOF file for reading
    file, err := os.Open(aofFilename)
    if err != nil {
        return fmt.Errorf("could not open AOF file: %v", err)
    }
    defer file.Close()

    // Read the file line by line
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        line := scanner.Text()

        // Check if the line contains a "SET" command
        if strings.HasPrefix(line, "SET") {
            parts := strings.Fields(line)
            if len(parts) == 3 {
                // Set the key-value pair with a TTL of one hour
                cache.Set(parts[1], parts[2], time.Hour, true)
            }
        }
    }

    // Handle any errors encountered while reading the AOF file
    if err := scanner.Err(); err != nil {
        return fmt.Errorf("error reading AOF file: %v", err)
    }

    return nil
}

/*
HandleConnection processes commands sent by the client over a network connection.
It reads commands from the client, processes them, and writes the appropriate response back to the client.
Commands are expected in the format "COMMAND key value" for SET commands, or "COMMAND key" for GET commands.
*/
func (s *Server) HandleConnection(conn net.Conn) {
    defer conn.Close()              // Ensure the connection is closed after processing
    reader := bufio.NewReader(conn) // Create a reader to read data from the connection

    // Continuously read and process commands from the client
    for {
        // Read the incoming message from the client
        message, err := reader.ReadString('\n')
        if err != nil {
            log.Println(err)
            return
        }

        // Clean up the message by trimming spaces
        message = strings.TrimSpace(message)
        // Split the message into command tokens
        tokens := strings.Split(message, " ")

        // Ensure there are at least two tokens (command and key)
        if len(tokens) < 2 {
            conn.Write([]byte("Invalid command\n"))
            continue
        }

        // Convert the command to uppercase for consistency
        cmd := strings.ToUpper(tokens[0])

        // Process different Redis commands
        switch cmd {
        case "SET":
            // Handle the SET command (set a key-value pair)
            if len(tokens) < 3 {
                conn.Write([]byte("Usage: SET key value\n"))
                continue
            }

            // Set the key-value pair in the cache with a TTL of 1 hour
            s.cache.Set(tokens[1], tokens[2], time.Hour, false)
            conn.Write([]byte("OK\n"))

        case "GET":
            // Handle the GET command (retrieve a value by key)
            key := tokens[1]
            if item, exists := s.cache.Get(key); !exists {
                // Respond with "nil" if the key doesn't exist in the cache
                conn.Write([]byte("nil\n"))
            } else {
                // Respond with the value if the key exists in the cache
                conn.Write([]byte(item + "\n"))
            }

        default:
            // Handle unknown commands
            conn.Write([]byte("Unknown Command\n"))
        }
    }
}

/*
Start listens for incoming connections on the specified address and port.
It accepts client connections and spawns a goroutine to handle each connection concurrently.
*/
func (s *Server) Start(address string) {
    // Set up the server to listen on the specified address (TCP port)
    listener, err := net.Listen("tcp", address)
    if err != nil {
        panic(err)
    }

    defer listener.Close() // Ensure the listener is closed when the function exits

    fmt.Println("Cache server running on port", address) // Log that the server has started

    // Continuously accept incoming client connections
    for {
        // Accept an incoming connection
        conn, err := listener.Accept()
        if err != nil {
            log.Println(err)
            continue // Skip any errors in accepting connections
        }

        // Handle the connection concurrently by spawning a goroutine
        go s.HandleConnection(conn)
    }
}

/*
Main function initializes the server and starts it to listen on port 6379.
It specifies the AOF filename for persistence and begins handling incoming client connections.
*/
func main() {
    // Specify the AOF file for persistence
    aofFilename := "cache.aof"

    // Initialize the server with AOF persistence
    server, err := NewServer(aofFilename)
    if err != nil {
        log.Fatalf("Error initializing server: %v", err)
    }

    // Start the server on port 6379
    server.Start(":6379")
}

Thanks for following along, and happy coding!

Learn Python by Building a Lisp Interpreter

Vishal Shukla — Wed, 22 Jan 2025 15:32:26 +0000

Introduction to Lisp

Lisp, an acronym for list processing, is a functional programming language that was designed for easy manipulation of data strings. Lisp is renowned for its simplicity and elegance in symbolic computation. Lisp programs are composed of expressions (lists), making it uniquely suited for recursive operations and manipulation of symbolic data.

In this blog, we’ll build a minimalist Lisp interpreter in Python. By the end of this guide, you’ll have a working interpreter capable of evaluating basic Lisp expressions, defining functions, and performing conditional logic.

Example of Lisp Syntax

Below is a simple Lisp program to calculate the cube of a number:

(define square (lambda (x) (* x x)))
(square 4) 
Output: 16

In this example:

define is used to create new variables or functions in Lisp. It associates a name with a value or a function, allowing you to reference it later in your code.
lambda is used to create anonymous functions in Lisp. These are functions without a predefined name. Instead of defining them upfront, you can create them dynamically and assign them to variables (or pass them around).

If you'd like to dive deeper into Lisp programming, a great starting point is this turotial.

Steps to Implement a Lisp Interpreter

Create a new file named lis.py. This will be the main Python script for the interpreter.

1. Tokenization:

In this step, the input string is split into smaller, meaningful pieces called tokens. For example, an expression like (+ 1 2) is broken into the list ["(", "+", "1", "2", ")"]. This process makes analyzing and manipulating the input easier for further processing.

def tokenize(source):
    """
    Tokenizes the input source string by:
    - Replacing opening and closing parentheses with space-padded versions to treat them as separate tokens.
    - Splitting the modified string by spaces to get a list of tokens.

    Args:
    - source (str): The source code to tokenize.

    Returns:
    - List of strings: A list of tokens that represent the source code.
    """
    return source.replace("(", " ( ").replace(")"," ) ").split()

2. Parsing:

After tokenization, the tokens are converted into a structured format, typically a nested list. For instance, ["(", "+", "1", "2", ")"] becomes ["+", 1, 2]. This structured representation allows for easier evaluation of the expression by mapping it to a tree-like structure known as an Abstract Syntax Tree (AST).

def read_from_tokens(tokens):
    """
    Reads from the list of tokens and recursively builds the corresponding Abstract Syntax Tree (AST).
    Handles parentheses to construct nested lists, and converts tokens to the corresponding atomic values (e.g., numbers or symbols).

    Args:
    - tokens (list): A list of tokens to process.

    Returns:
    - The corresponding expression represented by the tokens.

    Raises:
    - EOFError: If the token list is empty unexpectedly.
    - SyntaxError: If there is an unexpected closing parenthesis.
    """
    if not tokens:
        raise EOFError("Unexpected EOF while reading")

    token = tokens.pop(0)
    if token == '(':
        res = []
        while tokens[0] != ')':
            res.append(read_from_tokens(tokens))
        tokens.pop(0)
        return res
    elif token == ')':
        return SyntaxError("Unexpected ')'")  # Error if closing parenthesis appears too early.
    else:
        return atom(token)

def atom(token):
    """
    Attempts to convert a token to an integer, float, or leave it as a symbol if it's neither.

    Args:
    - token (str): The token to convert.

    Returns:
    - The converted token, which could be an integer, float, or the original string.
    """
    try:
        return int(token)
    except ValueError:
        try:
            return float(token)
        except ValueError:
            return token

 def parse(source):
    """
    Parses the source code into an Abstract Syntax Tree (AST).

    Args:
    - source (str): The source code to parse.

    Returns:
    - The Abstract Syntax Tree (AST) representing the source code.
    """
    return read_from_tokens(tokenize(source))

3. Environment Setup:

The environment acts as a dictionary where variable names and functions are stored. It includes built-in functions like +, -, *, /, and others, as well as user-defined variables and functions. This environment allows the interpreter to resolve symbols (like x or +) when used in expressions.

class Env(dict):
    """
    Represents an environment that holds variables and their values.
    It also links to an outer environment for variable resolution in nested scopes.

    Attributes:
    - params (tuple): The list of parameter names.
    - args (tuple): The values of the parameters.
    - outer (Env): The outer environment, for scoping purposes.
    """
    def __init__(self, params=(), args=(), outer=None):
        self.update(zip(params, args))
        self.outer = outer

    def find(self, var):
        """
        Finds the environment in which a variable is defined.

        Args:
        - var (str): The variable name to search for.

        Returns:
        - The environment that contains the variable.
        """
        return self if (var in self) else self.outer.find(var)

def standard_env():
    """
    Creates a standard environment for evaluating Scheme-like expressions.
    The environment contains built-in mathematical functions and operators, such as +, -, *, etc.

    Returns:
    - An Env object containing the standard functions and operators.
    """
    env = Env()
    env.update(vars(math))  # Import standard math functions like sin, cos, sqrt, etc.
    env.update({
        '+': op.add, '-': op.sub, '*': op.mul, '/': op.truediv, 
        '>': op.gt, '<': op.lt, '>=': op.ge, '<=': op.le, '=': op.eq, 
        'abs': abs,
        'append': op.add,  
        'begin': lambda *x: x[-1],
        'car': lambda x: x[0],
        'cdr': lambda x: x[1:], 
        'cons': lambda x, y: [x] + y,
        'eq?': op.is_, 
        'equal?': op.eq, 
        'length': len, 
        'list': lambda *x: list(x), 
        'list?': lambda x: isinstance(x, list), 
        'map': map,
        'max': max,
        'min': min,
        'not': op.not_,
        'null?': lambda x: x == [], 
        'number?': lambda x: isinstance(x, (int, float)),   
        'procedure?': callable,
        'round': round,
        'symbol?': lambda x: isinstance(x, str),
    })
    return env

global_env = standard_env()

Why Do We Need the `Env` Class?

The Env class is crucial because it acts as the backbone of the interpreter, managing variables, functions, and scope. Here’s why it’s needed:

Variable Binding

In Lisp, variables can be defined and used later in expressions. The Env class serves as a dictionary that stores variable names as keys and their corresponding values as entries.

Example:
```
(define x 10)  
(+ x 5)  ; Result: 15
```
The define operation associates x with 10. When x is referenced later, the Env class retrieves its value.
Function Storage

Lisp allows defining custom functions. These functions are stored in the environment and can be called later. The Env class keeps track of these functions and their corresponding implementations.

Example:
```
    (define square (lambda (x) (* x x)))  
    (square 4)  ; Result: 16
```
Scoped Resolution

Lisp supports nested scopes. For example, variables defined within a function should not overwrite global variables. The Env class supports this by linking to an outer environment, allowing scoped variable resolution.

Example:
```
(define x 10)  
(define square (lambda (x) (* x x)))  
(square 4)  ; Result: 16  
x           ; Result: 10
```
Built-in Functions

The Env class includes a standard environment containing built-in operations (e.g., +, -, *, /, sin, cos) so that basic computations can be performed immediately.

4. Evaluation:

The core logic of the interpreter, where parsed expressions are computed:

Numbers (e.g., 1, 2.5) and variables (e.g., x, y) evaluate to their respective values.
Function calls (e.g., (+ 1 2)) are recursively evaluated. The operator (+) and arguments (1, 2) are resolved and computed using the environment.

 class Procedure(object):
    """
    Represents a user-defined function in the Lisp environment.

    Attributes:
    - params (tuple): The parameters of the function.
    - body (list): The body of the function (a list of expressions to evaluate).
    - env (Env): The environment in which the function was created.
    """
    def __init__(self, params, body, env):
        self.params, self.body, self.env = params, body, env

    def __call__(self, *args):
        """
        Calls the function with provided arguments, creating a new environment for the call.

        Args:
        - args (tuple): The arguments to pass to the function.

        Returns:
        - The result of evaluating the function's body in the new environment.
        """
        local_env = Env(self.params, args, self.env)
        return eval(self.body, local_env)

def eval(ast, env=global_env):
    """
    Evaluates an Abstract Syntax Tree (AST) in the given environment.

    Args:
    - ast: The AST to evaluate.
    - env (Env): The environment in which to evaluate the AST.

    Returns:
    - The result of evaluating the AST.
    """
    match ast:
        case str() as symbol:
            return env.find(symbol)[symbol]  

        case int() | float() as value:
            return value  

        case ["quote", value]:
            return value  

        case ["define", variable, value]:
            evaluated_value = eval(value, env)  
            env[variable] = evaluated_value  
            return evaluated_value

        case ["if", condition, operation, alternative]:
            conditional_value = eval(condition, env)  
            if conditional_value:  
                return eval(operation, env)
            else: 
                return eval(alternative, env)

        case ["lambda", params, body]:
            return Procedure(params, body, env)  

        case [operator, *args]:
            operator_value = eval(operator, env) 
            operands = [eval(operand, env) for operand in args]  
            return operator_value(*operands)

5. REPL (Read-Eval-Print Loop):

The REPL is an interactive shell that allows users to input and execute Lisp commands in real-time. It reads user input (e.g., (define x 10)), evaluates it (e.g., assigns 10 to x), and prints the result.

def repl(prompt="lis.py> "):
    """
    The Read-Eval-Print Loop (REPL) for evaluating Lisp expressions.

    Args:
    - prompt (str): The prompt to display to the user.
    """
    while True:
        try:
            source = input(prompt)  
            if source.strip().lower() == "exit":
                print("Exiting")  
                break
            print(eval(parse(source)))  
        except Exception as e:
            print(f"Error: {e}")

Conclusion

By following the steps above, you've created a basic Lisp interpreter that can handle arithmetic expressions, define functions, and even handle conditional logic. This is just a simple version, but as you explore further, you can expand it with more features like advanced error handling, better scoping, and additional built-in functions.

If you're interested in diving deeper into the world of Lisp and learning more advanced concepts, I highly recommend checking out Peter Norvig’s Lisp Interpreter Tutorial, which was an excellent resource for building the interpreter in this guide.

Build an OTP-Based Authentication Server with Go: Part 4 - File Uploads and Graceful Shutdown

Vishal Shukla — Fri, 17 Jan 2025 08:18:22 +0000

We will implement a route to upload files, simplify our workflow by defining commands using a Makefile, and add a graceful shutdown mechanism to our server. This ensures the server does not stop abruptly and completes ongoing tasks before shutting down.

Graceful Shutdown

Create a new cmd/api/server.go file to manage all server-related functionalities in a separate file. We will create a new Go routine that listens for termination signals, and upon receiving one, it will gracefully shut down the server.

I have added comments to make it easier to understand what each part does.

package main

import (
    "context"
    "errors"
    "fmt"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func (app *application) serve(router http.Handler) error {
    /**
     * Initialize a new HTTP server instance with configurations defined in the application config.
     * The server includes timeouts for idle, read, and write operations to handle connections gracefully.
     * The "Handler" chain applies middleware for panic recovery and authentication.
     */
    srv := &http.Server{
        Addr:         fmt.Sprintf(":%d", app.config.port),
        Handler:      app.recoverPanic(app.authenticate(router)),
        IdleTimeout:  time.Minute,
        ReadTimeout:  10 * time.Second,
        WriteTimeout: 30 * time.Second,
    }

    /**
     * Create a channel to capture errors that might occur during the shutdown process.
     * This channel ensures the serve function properly waits for shutdown tasks to complete.
     */
    shutdownError := make(chan error)

    /**
     * Start a goroutine to listen for termination signals (e.g., SIGINT, SIGTERM).
     * Upon receiving a signal, the server initiates a graceful shutdown sequence.
     */
    go func() {
        quit := make(chan os.Signal, 1)
        signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)

        s := <-quit
        app.logger.Printf("Received %v, starting shutdown", s.String())

        ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
        defer cancel()

        err := srv.Shutdown(ctx)
        if err != nil {
            shutdownError <- err
        }

        /**
         * Wait for any background tasks to complete.
         * The application's WaitGroup (app.wg) ensures that no tasks are left running.
         */
        app.logger.Printf("completing background tasks: port %d", app.config.port)
        app.wg.Wait()

        shutdownError <- nil
    }()

    /**
     * Log the server startup details, including the configured port and environment mode.
     * The server will start listening for incoming connections.
     */
    app.logger.Printf("Starting server on port %d in %s mode", app.config.port, app.config.env)
    err := srv.ListenAndServe()

    /**
     * If the server stops unexpectedly (not due to a shutdown signal), return the error.
     */
    if !errors.Is(err, http.ErrServerClosed) {
        return err
    }

    /**
     * Wait for the shutdown process to complete and capture any errors that occur.
     * If there is an error during shutdown, return it wrapped with additional context.
     */
    err = <-shutdownError
    if err != nil {
        return fmt.Errorf("server shutdown error: %w", err)
    }

    /**
     * Log that the server stopped successfully and return nil to indicate no errors.
     */
    app.logger.Println("server stopped :)")

    return nil
}

Let's use this serve function in our main.go file. Remove the srv.ListenAndServe block and replace it with this.

err = app.serve(router)
    if err != nil {
        logger.Fatal(err, nil)
    }

Try shutting down the server using the Ctrl+C command, and you’ll notice that it waits for ongoing tasks to complete before shutting down. To test this, you can use the retry mechanism we implemented when sending OTPs. Remove the credentials so it keeps retrying, and then try shutting down the server midway. You’ll observe that it waits for the retries to finish before shutting down.

Automating Commands with Makefile

Next, we need to create new database migrations, but if you've been following along, you might have noticed how tedious it is to write the same commands repeatedly. Let's make this easier by automating it with a Makefile.

In the root directory of your project, open the Makefile and add the following content:

# Include variables from the .envrc file
include .envrc

## help: print this help message
.PHONY: help
help:
    @echo 'Usage:'
    @sed -n 's/^##//p' ${MAKEFILE_LIST} | column -t -s ':' | sed -e 's/^/ /'

.PHONY: confirm
confirm:
    @echo -n 'Are you sure? [y/N] ' && read ans && [ $${ans:-N} = y ]

## run/api: run the cmd/api application
.PHONY: run/api
run/api:
    go run ./cmd/api

## db/psql: connect to the database using psql
.PHONY: db/psql
db/psql:
    psql ${GREENLIGHT_DB_DSN}

## db/migrations/new name=$1: create a new database migration
.PHONY: db/migrations/new
db/migrations/new:
    @echo "Migrating ${name}"
    migrate create -seq -ext=.sql -dir=./migrations ${name}

## db/migrations/up: apply all up database migrations
.PHONY: db/migrations/up
db/migrations/up: confirm
    @echo "Running migrations"
    migrate -path=./migrations -database=${GREENLIGHT_DB_DSN} up


## audit: tidy and vendor dependencies and format, vet and test all code
.PHONY: audit
audit: vendor
    @echo 'Formatting code...'
    go fmt ./...
    @echo 'Vetting code...'
    go vet ./...
    staticcheck ./...
    @echo 'Running tests...'
    go test -race -vet=off ./...
## vendor: tidy and vendor dependencies
.PHONY: vendor
vendor:
    @echo 'Tidying and verifying module dependencies...'
    go mod tidy
    go mod verify
    @echo 'Vendoring dependencies...'
    go mod vendor

## build/api: build the cmd/api application
.PHONY: build/api
build/api:
    @echo 'Building cmd/api...'
    go build -o=./bin/api ./cmd/api
    GOOS=linux GOARCH=amd64 go build -o=./bin/linux_amd64/api ./cmd/api

Run a Make Command

Once in the project directory, you can run any of the available commands defined in the Makefile. Here are a few examples:

To display the help message and see all available commands:


make help

To run your Go application:


make run/api

To create a new database migration (you will need to pass a name for the migration):


make db/migrations/new name=my_migration

To apply all pending migrations:


make db/migrations/up

To format, vet, and test your code:


make audit

To build the Go application for your default platform:


make build/api

Add More Commands

You can add more commands to the Makefile as needed, following the structure provided. Each task is defined with a .PHONY declaration (indicating that it's not a real file target), followed by the task name and the commands to execute.

This simple process helps you automate tasks, manage your Go project efficiently, and improve productivity.

File upload and tracking

Next, we’ll create a new table in the database to track all files and their associated metadata.

Run the following command to create a new migration:

 make db/migrations/new name=create-creative

Then, add the following SQL code in the 000003_create-creative.up.sql and 000003_create-creative.down.sql files, respectively:

CREATE TABLE IF NOT EXISTS creatives (
    id bigserial PRIMARY KEY, 
    user_id bigint NOT NULL REFERENCES users on DELETE CASCADE,
    creative_url text NOT NULL,
    scheduled_at DATE NOT NULL,
    created_at timestamp(0) with time zone NOT NULL DEFAULT NOW()
);

DROP  TABLE  IF  EXISTS creatives;

Next, we’ll integrate this into our central repository model, following the same approach we’ve been using. Create a new file called internal/data/creatives.go and add the following code:

package data

import (
    "context"
    "database/sql"
    "time"

    "github.com/lib/pq"
)

type Creative struct {
    ID          int64     `json:"id"`
    UserID      int64     `json:"-"`
    CreativeURL string    `json:"creative_url"`
    ScheduledAt time.Time `json:"scheduled_at"`
    CreatedAt   time.Time `json:"created_at"`
}

type CreativeModel struct {
    DB *sql.DB
}

func (c *CreativeModel) Insert(creative *Creative) error {
    query := `INSERT INTO creatives (user_id, creative_url, scheduled_at)
            VALUES ($1, $2, $3)
            RETURNING id, created_at`

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    args := []interface{}{creative.UserID, creative.CreativeURL, creative.ScheduledAt}
    err := c.DB.QueryRowContext(ctx, query, args...).Scan(&creative.ID, &creative.CreatedAt)

    if err != nil {
        return err
    }

    return nil
}

func (c *CreativeModel) GetScheduledCreatives() (map[string][]Creative, error) {
    query := `
        SELECT id, user_id, creative_url, scheduled_at, created_at 
        FROM creatives 
        WHERE scheduled_at = ANY($1)
    `

    dates := []time.Time{
        time.Now().Truncate(24 * time.Hour),
        time.Now().AddDate(0, 0, 1).Truncate(24 * time.Hour),
    }

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    rows, err := c.DB.QueryContext(ctx, query, pq.Array(dates))
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    creatives := map[string][]Creative{
        "today":    {},
        "tomorrow": {},
    }

    for rows.Next() {
        var creative Creative
        err := rows.Scan(&creative.ID, &creative.UserID, &creative.CreativeURL, &creative.ScheduledAt, &creative.CreatedAt)
        if err != nil {
            return nil, err
        }

        if creative.ScheduledAt.Equal(dates[0]) {
            creatives["today"] = append(creatives["today"], creative)
        } else if creative.ScheduledAt.Equal(dates[1]) {
            creatives["tomorrow"] = append(creatives["tomorrow"], creative)
        }
    }

    if err = rows.Err(); err != nil {
        return nil, err
    }

    return creatives, nil
}

Controller for File Upload and Tracking

Next, we'll create a new file cmd/api/creative.go where we’ll define two handlers: one to upload a file and another to retrieve scheduled files for today.

Create the file and paste the following code:

package main

import (
    "fmt"
    "io"
    "net/http"
    "os"
    "path"
    "strings"
    "time"

    "github.com/google/uuid"
    "github.com/vishaaxl/cheershare/internal/data"
)

const MaxFileSize = 10 << 20

/**
 * generateUUIDFilename generates a unique filename based on a UUID.
 * It uses the original file's extension to keep the file format intact.
 * This helps avoid naming conflicts when storing uploaded files.
 */
func generateUUIDFilename(filePath string) string {
    ext := path.Ext(filePath)
    return fmt.Sprintf("%s%s", uuid.New().String(), ext)
}

/**
 * isImageFile checks whether the uploaded file is an image.
 * It checks the file extension to ensure that only .jpg, .jpeg, .png, and .gif files are allowed.
 */
func isImageFile(filename string) bool {
    allowedExtensions := []string{".jpg", ".jpeg", ".png", ".gif"}
    ext := strings.ToLower(path.Ext(filename))

    for _, validExt := range allowedExtensions {
        if ext == validExt {
            return true
        }
    }

    return false
}

/**
 * uploadFile handles the file upload logic.
 * It parses the incoming form data, checks for errors, and saves the file to disk.
 * The method also ensures that only images are uploaded by checking the file type.
 */
func (app *application) uploadFile(r *http.Request) (string, error) {
    err := r.ParseMultipartForm(MaxFileSize)
    if err != nil {
        /**
         * If parsing the form fails (e.g., file size exceeds limit or form is malformed),
         * return an empty string and the error to notify the caller.
         */
        return "", err
    }

    /**
     * Retrieve the uploaded file from the form data using the key "file".
     * This returns the file object, its header (containing metadata like filename), and an error if any.
     */
    file, header, err := r.FormFile("file")
    if err != nil {
        /**
         * If there was an error retrieving the file (e.g., missing file in the request),
         * return an empty string and the error to notify the caller.
         */
        return "", err
    }
    defer file.Close()

    /**
     * Check if the uploaded file is a valid image (using the file extension).
     * The isImageFile function verifies that the file is one of the allowed image types (e.g., .jpg, .png).
     */
    if !isImageFile(header.Filename) {
        return "", fmt.Errorf("invalid file type: only images are allowed")
    }

    /**
     * Generate a unique filename for the uploaded file using a UUID to avoid naming conflicts.
     * The generateUUIDFilename function ensures that each uploaded file gets a unique name,
     * while retaining the file's original extension.
     */
    uploadDir := "./uploads"
    destinationFilePath := fmt.Sprintf("%s/%s", uploadDir, generateUUIDFilename(header.Filename))

    /**
     * Create the destination file in the specified upload directory.
     * The os.Create function returns a file pointer for the newly created file.
     */
    destinationFile, err := os.Create(destinationFilePath)
    if err != nil {
        return "", err
    }
    defer destinationFile.Close()

    /**
     * Copy the contents of the uploaded file to the newly created destination file.
     * The io.Copy function writes from the source (file) to the destination (destinationFile).
     */
    _, err = io.Copy(destinationFile, file)
    if err != nil {
        return "", err
    }

    return destinationFilePath, nil
}

/**
 * uploadCreativeHandler handles the HTTP request for uploading a creative file.
 * It processes the file upload, checks for errors, and returns a response with the uploaded file path.
 */
func (app *application) uploadCreativeHandler(w http.ResponseWriter, r *http.Request) {
    /**
     * Extract the "scheduled_at" parameter from the request form data.
     * This represents the date when the creative will be scheduled.
     * If the "scheduled_at" parameter is missing, respond with a 400 Bad Request error.
     */
    scheduledAtStr := r.FormValue("scheduled_at")
    if scheduledAtStr == "" {
        app.errorResponse(w, http.StatusBadRequest, "scheduled_at is required")
        return
    }

    scheduledAt, err := time.Parse("2006-01-02", scheduledAtStr)
    if err != nil {
        app.errorResponse(w, http.StatusBadRequest, "invalid date format for scheduled_at")
        return
    }

    /**
     * Validate that the scheduled date is not in the past.
     * If the date is earlier than the current date, respond with a 400 Bad Request error.
     */
    if scheduledAt.Before(time.Now()) {
        app.errorResponse(w, http.StatusBadRequest, "cannot set scheduled_at before today")
        return
    }

    /**
     * Handle the file upload using the app.uploadFile method.
     * If the file upload fails, respond with a 400 Bad Request error containing the error message.
     */
    uploadedFile, err := app.uploadFile(r)
    if err != nil {
        app.errorResponse(w, http.StatusBadRequest, err.Error())
        return
    }

    creative := &data.Creative{
        CreativeURL: uploadedFile,
        ScheduledAt: scheduledAt,
        UserID:      app.contextGetUser(r).ID,
    }

    err = app.models.Creative.Insert(creative)
    if err != nil {
        app.logger.Println("Error saving creative:", err)
        app.errorResponse(w, http.StatusInternalServerError, "failed to save creative")
        return
    }

    app.writeJSON(w, http.StatusOK, envelope{"creative": creative}, nil)
}

func (app *application) getScheduledCreativesHandler(w http.ResponseWriter, r *http.Request) {
    scheduledCreatives, err := app.models.Creative.GetScheduledCreatives()
    if err != nil {

        app.errorResponse(w, http.StatusInternalServerError, "failed to fetch scheduled creatives")
        return
    }

    app.writeJSON(w, http.StatusOK, envelope{"scheduled_creatives": scheduledCreatives}, nil)
}

Instead of checking for the existence of the ./uploads folder every time a request is made, we will perform this check when the server starts. If the directory doesn't exist, we will create it.

Add the following code to your main.go or server initialization logic and add the two new methods to upload and get the files:

uploadDir := "./uploads"
if _, err := os.Stat(uploadDir); os.IsNotExist(err) {
    err := os.MkdirAll(uploadDir, os.ModePerm)
    if err != nil {
        fmt.Println("Unable to create uploads directory:", err)
    }
}
...

router.HandlerFunc(http.MethodPost, "/upload-creative", app.requireAuthenticatedUser(app.uploadCreativeHandler))
router.HandlerFunc(http.MethodGet, "/scheduled", app.requireAuthenticatedUser(app.getScheduledCreativesHandler))

This concludes our Go backend development series. We've successfully implemented most of the essential features, with the exception of the payment functionality. Stay tuned for future updates where we will dive deeper into more advanced features!

Part 3: https://dev.to/vishaaxl/build-an-otp-based-authentication-server-with-go-part-3-2jc0

Building a String Calculator with Test-Driven Development (TDD): A Step-by-Step Guide

Vishal Shukla — Wed, 15 Jan 2025 09:46:41 +0000

We will implement a string calculator in Python using a test-driven development (TDD) approach. This means we will write tests for each feature before implementing the corresponding functionality.

You can refer to the link https://osherove.com/tdd-kata-1 as your checkpoints for implementing TDD. The link provides step-by-step instructions that you can follow.

Getting started

In your project folder, create two files: string_calculator.py and tests/test_string_calculator.py. We'll implement the features step by step. First, we need to create a StringCalculator class with an add method.

Step 1: Empty String Should Return "0"

Let's write the first test for our application using the unittest library. Open the tests/test_string_calculator.py file and start with the following code:

import unittest
from string_calculator import StringCalculator

class TestStringCalculator(unittest.TestCase):
    """Test suite for the StringCalculator class."""

    def setUp(self):
        """
        Create a new instance of StringCalculator for each test.
        Can use static method to avoid creating a new instance.
        """
        self.calculator = StringCalculator()

    def test_empty_string_returns_zero(self):
        """
        Test case: Adding an empty string should return 0.
        Input: "" 
        Expected Output: 0
        """
        self.assertEqual(self.calculator.add(""), 0)

Now, let's implement the StringCalculator class in the string_calculator.py file:

class StringCalculator:
    def add(self, numbers:str):
        if not numbers:
            return 0

To run the tests, follow these steps:

Ensure that you are in the project directory where your string_calculator.py and tests/test_string_calculator.py files are located.
Open your terminal or command prompt.
Run the following command to execute the tests:

python -m unittest discover tests

This command will automatically discover and run all tests within the tests folder.

Expected Output:

You should see something like this if the test passes:



----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

If everything is set up correctly and the test case passes, it means your implementation for handling an empty string is working as expected.

Step 2: Adding One or Two Numbers Should Return Their Sum

We need to update the method to handle the case where there is only one number or two numbers in the input string, and it should return their sum. For an empty string, the method should return 0.

Writing the Test

Open the tests/test_string_calculator.py file and add the following test cases to cover all the scenarios:

    def test_add_single_number(self):
        """
        Test case: Adding a single number should return the number itself.
        Input: "1"
        Expected Output: 1
        """
        self.assertEqual(self.calculator.add("1"), 1)

    def test_add_two_numbers(self):
        """
        Test case: Adding two numbers should return their sum.
        Input: "1,2"
        Expected Output: 3
        """
        self.assertEqual(self.calculator.add("1,2"),3)

Implementing the Code

Now, update the add method in the string_calculator.py file to handle the addition of one or two numbers:

class StringCalculator:
    def add(self, numbers:str):
        if not numbers:
            return 0
        '''
        Split the string by commas, convert each value to an integer, 
        and sum them up
        '''
        numbers_list = map(int,numbers.split(',')) 
        return sum(numbers_list)

You can test the code again by following the previous steps.

Step 3 : Handling Multiple Numbers

We'll write a test case to check if the method can handle multiple numbers separated by commas.

Writing the Test

Open the tests/test_string_calculator.py file and add a test case to handle multiple numbers:

def test_add_multiple_numbers(self):
    """
    Test case: Adding multiple numbers should return their sum.
    Input: "1,2,3,4,5"
    Expected Output: 15
    """
    self.assertEqual(self.calculator.add("1,2,3,4,5"), 15)

The functionality has already been implemented, so we can proceed to test the code and then move on to the next step.

Step 4: Handling New Lines Between Numbers

Now, we need to enhance the add method to handle new lines (\n) as valid separators between numbers, in addition to commas.

Writing the Test

Open the tests/test_string_calculator.py file and add a test case to check if the method correctly handles new lines as separators:

def test_add_numbers_with_newlines(self):
    """
    Test case: Adding numbers separated by newlines should return their sum.
    Input: "1\n2\n3"
    Expected Output: 6
    """
    self.assertEqual(self.calculator.add("1\n2\n3"), 6)

Implementing the Code

Next, update the add method in the string_calculator.py file to handle new lines (\n) as separators. We can modify the method to replace \n with commas, then split the string by commas.

Here's the updated code for the add method:

class StringCalculator:
    def add(self, numbers:str):
        if not numbers:
            return 0

        numbers = numbers.replace("\n", ",")

        '''
        Split the string by commas, convert each value to an integer, 
        and sum them up
        '''
        numbers_list = map(int,numbers.split(',')) 
        return sum(numbers_list)

You can test the code again by following the previous steps defined in step1.

Step 5: Handling Custom Delimiters

In this step, we will enhance the functionality further to allow custom delimiters. For instance, users should be able to specify a custom delimiter at the beginning of the string. For example:

The input string could start with // followed by a custom delimiter, e.g., //;\n1;2;3 should return 6.
We will support delimiters like //;\n1;2;3.

Writing the Test

Open the tests/test_string_calculator.py file and add a test case to handle the custom delimiter functionality:

def test_add_numbers_with_custom_delimiter(self):
    """
    Test case: Adding numbers separated by a custom delimiter should return their sum.
    Input: "//;\n1;2;3"
    Expected Output: 6
    """
    self.assertEqual(self.calculator.add("//;\n1;2;3"), 6)

Implementing the Code

To handle custom delimiters, update the add method to look for the delimiter in the input string. The delimiter should be specified at the beginning of the string after //.

Here’s the updated add method:

class StringCalculator:
    def add(self, numbers:str):
        if not numbers:
            return 0

        if numbers.startswith("//"):
            delimiter_end_index = numbers.index("\n")
            delimiter = numbers[2:delimiter_end_index]
            numbers = numbers[delimiter_end_index + 1:]
            numbers = numbers.replace(delimiter,",")

        numbers = numbers.replace("\n", ",")

        '''
        Split the string by commas, convert each value to an integer, 
        and sum them up
        '''
        numbers_list = map(int,numbers.split(',')) 
        return sum(numbers_list)

Step 6: Handling Negative Numbers

In this step, we need to modify the add method to handle negative numbers. When a negative number is passed, it should throw an exception with the message "negatives not allowed", and include the negative numbers that were passed.

Writing the Test

Open the tests/test_string_calculator.py file and add a test case to handle the negative number exception:

def test_add_negative_numbers(self):
    """
    Test case: Adding numbers with negative numbers should raise an exception.
    Input: "-1,2,3"
    Expected Output: "Negative numbers are not allowed: -1"
    """
    with self.assertRaises(ValueError) as e:
        self.calculator.add("-1,2,-3")
    self.assertEqual(str(e.exception), "Negative numbers are not allowed: -1, -3")

Implementing the Code

Now, modify the add method to check for negative numbers and raise a ValueError with the appropriate message.

Here's the updated add method:

class StringCalculator:
    def add(self, numbers:str):
        if not numbers:
            return 0

        if numbers.startswith("//"):
            delimiter_end_index = numbers.index("\n")
            delimiter = numbers[2:delimiter_end_index]
            numbers = numbers[delimiter_end_index + 1:]
            numbers = numbers.replace(delimiter,",")

        numbers = numbers.replace("\n", ",")



        numbers_list = numbers.split(",")
        negatives = []

        for num in numbers_list:
            number = int(num)
            if number < 0:
                negatives.append(number)

        if negatives:
            raise ValueError(f"negative numbers are not allowed: {', '.join(map(str, negatives))}")

        return sum(map(int,numbers_list))

Step 7: Counting Add Method Calls

In this step, we will add a method called GetCalledCount() to the StringCalculator class that will return how many times the add() method has been invoked. We will follow the TDD process by writing a failing test first, and then implementing the feature.

Writing the Test

Start by adding a test case for the GetCalledCount() method. This test should check that the method correctly counts the number of times add() is called.

Open the tests/test_string_calculator.py file and add the following test:

def test_get_called_counts(self):
    """
    Test case: Calling get_called_counts method should return the number of times add method was called.
    Input: Multiple add calls
    Expected Output: 2 (if add() was called 2 times)
    """
    self.calculator.add("1,2,3")
    self.calculator.add("1,2")
    self.assertEqual(self.calculator.get_called_count(), 2)

Implementing the Code

Now, implement the GetCalledCount() method in the StringCalculator class. This method will need to keep track of how many times add() has been invoked.

Here’s the updated StringCalculator class:

class StringCalculator:
    def __init__(self):
        self.called_count = 0
    def add(self, numbers:str):
        if not numbers:
            return 0

        if numbers.startswith("//"):
            delimiter_end_index = numbers.index("\n")
            delimiter = numbers[2:delimiter_end_index]
            numbers = numbers[delimiter_end_index + 1:]
            numbers = numbers.replace(delimiter,",")

        numbers = numbers.replace("\n", ",")



        numbers_list = numbers.split(",")
        negatives = []

        for num in numbers_list:
            number = int(num)
            if number < 0:
                negatives.append(number)

        if negatives:
            raise ValueError(f"negative numbers are not allowed: {', '.join(map(str, negatives))}")

        self.called_count += 1
        return sum(map(int,numbers_list))

    def get_called_count(self):
        return self.called_count

Step 8 & 9: Ignore Numbers Greater Than 1000 and Handle Custom Delimiters of Any Length

In this step, we will implement two requirements:

Numbers greater than 1000 should be ignored in the sum.
Custom delimiters can be of any length, with the format //[delimiter]\n, and the method should handle them.

We will first write the tests for both of these requirements, then implement the functionality in the StringCalculator class.

Writing the Tests

Add the following tests for both the ignore numbers greater than 1000 and handling custom delimiters of any length. Open the tests/test_string_calculator.py file and add the following:

def test_add_numbers_greater_than_1000(self):
    """
    Test case: Numbers greater than 1000 should be ignored when calculating the sum.
    Input: "1,1001,2,3"
    Expected Output: 6
    """
    self.assertEqual(self.calculator.add("1,1001,2,3"), 6)

def test_add_numbers_with_custom_delimiter_long_length(self):
    """
    Test case: Adding numbers separated by a custom delimiter with a long length should return their sum.
    Input: "//[***]\n1***2***3"
    Expected Output: 6
    """
    self.assertEqual(self.calculator.add("//[***]\n1***2***3"), 6)

Implementing the Code

Now, implement the functionality in the StringCalculator class. This will include:

Ignoring numbers greater than 1000.
Handling custom delimiters of any length.

Here’s the updated StringCalculator class:

class StringCalculator:
    def __init__(self):
        self.called_count = 0
    def add(self, numbers:str):
        if not numbers:
            return 0

        if numbers.startswith("//"):
            delimiter_end_index = numbers.index("\n")
            delimiter = numbers[2:delimiter_end_index]

            if delimiter.startswith("[") and delimiter.endswith("]"):
                delimiter = delimiter[1:-1]

            numbers = numbers[delimiter_end_index + 1:]
            numbers = numbers.replace(delimiter,",")

        numbers = numbers.replace("\n", ",")



        numbers_list = numbers.split(",")
        negatives = []
        numbers_to_add = []

        for num in numbers_list:
            number = int(num)
            if number < 0:
                negatives.append(number)
            elif number > 1000:
                continue
            else:
                numbers_to_add.append(number)


        if negatives:
            raise ValueError(f"negative numbers are not allowed: {', '.join(map(str, negatives))}")

        self.called_count += 1
        return sum(map(int, numbers_to_add))

    def get_called_count(self):
        return self.called_count

Step 10: Multiple Delimiters Support

In this step, we will modify the add() method to support multiple delimiters of any length. This will allow us to handle cases where there are multiple delimiters in the format //[delimiter1][delimiter2]\n.

Writing the Test

Start by adding a test case to check for multiple delimiters. Open the tests/test_string_calculator.py file and add the following test:

def test_add_numbers_with_multiple_delimiters(self):
    """
    Test case: Multiple delimiters of any length can be used.
    Input: "//[***][%%]\n1***2%%3"
    Expected Output: 6
    """
    self.assertEqual(self.calculator.add("//[***][%%]\n1***2%%3"), 6)

Implementing the Code

Now, modify the add() method to handle multiple delimiters. The delimiters will be passed inside [], and we need to support handling multiple delimiters in the format //[delimiter1][delimiter2]\n.

Here's the updated StringCalculator class to support this:

import re

class StringCalculator:
    """
    A class to implement a string calculator that can add numbers from a given string.
    The calculator supports multiple delimiters, new lines between numbers, and backward compatibility with the older format.

    Attributes:
    called_count (int): The count of how many times the add() method was called.
    """

    def __init__(self):
        """
        Initializes a StringCalculator instance and sets the called count to 0.
        """
        self.called_count = 0

    def add(self, numbers: str):
        """
        Adds numbers from the given string, supporting custom delimiters, multiple delimiters, 
        ignoring numbers greater than 1000, and handling negative numbers with an exception.

        The input string can use the following formats:
        1. Comma-separated values (e.g., "1,2,3")
        2. New line-separated values (e.g., "1\n2\n3")
        3. Custom delimiters defined by the user in the format `//[delimiter]\n`.

        Additionally, the method:
        - Ignores numbers greater than 1000.
        - Throws a ValueError if any negative numbers are provided.

        Parameters:
        numbers (str): A string of numbers, optionally separated by commas, new lines, or custom delimiters.

        Returns:
        int: The sum of the numbers in the input string.

        Raises:
        ValueError: If any negative numbers are found, a ValueError is raised with the message 
                    "negative numbers are not allowed" followed by the negative numbers.

        Example:
        >>> calculator = StringCalculator()
        >>> calculator.add("1,2,3")
        6

        >>> calculator.add("//[***]\n1***2***3")
        6
        """
        if not numbers:
            return 0

        if numbers.startswith("//"):
            delimiter_end_index = numbers.index("\n")
            delimiter_section = numbers[2:delimiter_end_index]

            if delimiter_section.startswith("["):
                delimiters = re.findall(r'\[([^\]]+)\]', delimiter_section)
            else:
                delimiters = [delimiter_section]

            numbers = numbers[delimiter_end_index + 1:]

            for delimiter in delimiters:
                numbers = numbers.replace(delimiter, ",")

        numbers = numbers.replace("\n", ",")

        numbers_list = numbers.split(",")
        negatives = []
        numbers_to_add = []

        for num in numbers_list:
            number = int(num)
            if number < 0:
                negatives.append(number)
            elif number > 1000:
                continue  
            else:
                numbers_to_add.append(number)

        if negatives:
            raise ValueError(f"negative numbers are not allowed: {', '.join(map(str, negatives))}")

        self.called_count += 1
        return sum(map(int, numbers_to_add))

    def get_called_count(self):
        """
        Returns the number of times the add() method has been invoked.

        Returns:
        int: The count of times add() has been called.

        Example:
        >>> calculator = StringCalculator()
        >>> calculator.add("1,2")
        >>> calculator.get_called_count()
        1
        """
        return self.called_count

Testing It

Run the tests again to verify that everything works, including backward compatibility with the old format and support for the new multiple delimiters format:

python -m unittest discover tests

Expected Output

The tests should pass for both old and new formats:


..........
----------------------------------------------------------------------
Ran 10 tests in 0.003s

OK

Appreciate you following along with this TDD series! I hope you found it useful.

Build an OTP-Based Authentication Server with Go: Part 3

Vishal Shukla — Fri, 10 Jan 2025 09:20:51 +0000

In this part, we’ll implement the functionality to send OTPs to users using Twilio and introduce an optimized approach for sending OTPs asynchronously. Additionally, we’ll cover the creation of a token system to authenticate users.

Sending OTP with Twilio

The first task is to send the OTP to the user via Twilio's messaging API. Below is the function to handle the OTP sending process.

// sendOTPViaTwilio sends an OTP to the specified phone number using Twilio's messaging API.
//
// Parameters:
// - otp: The one-time password to be sent in the message body.
// - phoneNumber: The recipient's phone number without the country code.
//
// The function uses Twilio's Go SDK to create and send a message with the provided OTP.
// The `from` phone number is pre-configured to be a Twilio-registered number.
//
// Returns:
// - An error if the OTP could not be sent successfully.
// - nil if the message was sent without issues.
func (app *application) sendOTPViaTwilio(otp, phoneNumber string) error {
    client := twilio.NewRestClientWithParams(twilio.ClientParams{
        Username: os.Getenv("TWILIO_SID"),
        Password: os.Getenv("TWILIO_API_KEY"),
    })

    // Set up the parameters for the message.
    params := &api.CreateMessageParams{}
    params.SetBody(fmt.Sprintf(
        "Thank you for choosing Cheershare! Your one-time password is %v.",
        otp,
    ))
    params.SetFrom(os.Getenv("TWILIO_PHONE_NUMBER")) // Twilio-registered phone number.
    params.SetTo(fmt.Sprintf("+91%v", phoneNumber))  // Format recipient's number with country code.

    const maxRetries = 3 // Number of retries
    var lastErr error    // Stores the last error encountered

    // Attempt to send the message with retries.
    for attempt := 1; attempt <= maxRetries; attempt++ {
        resp, err := client.Api.CreateMessage(params)
        if err == nil {
            fmt.Printf("OTP sent successfully. Twilio response SID: %v\n", resp.Sid)
            return nil
        }

        // Log the error for debugging.
        lastErr = fmt.Errorf("attempt %d: failed to send OTP via Twilio: %w", attempt, err)
        fmt.Println(lastErr)

        time.Sleep(2 * time.Second)
    }

    return fmt.Errorf("all retries failed to send OTP via Twilio: %w", lastErr)
}

The retry mechanism in this function ensures the OTP is sent reliably. However, a key optimization is needed. Sending OTPs sequentially will slow down the server, so we’ll leverage goroutines to offload the sending process to a separate routine.

Improving Performance with Goroutines

We will use goroutines to run this task on a separate routine. Since goroutines are executed away from the main thread, we need to track the active goroutines using a sync.WaitGroup. This ensures that before shutting down, we can wait for all the active goroutines to finish. Let's update the application struct to include a sync.WaitGroup.

type application struct {
    wg     sync.WaitGroup //updated
    config config
    models data.Models

    logger *log.Logger
    cache  *redis.Client
}

Next, let’s create a helper function that takes a function as a parameter and runs it using the go keyword.

func (app *application) background(fn func()) {
    app.wg.Add(1)

    go func() {
        defer app.wg.Done()

        defer func() {
            if err := recover(); err != nil {
                app.logger.Printf("Error in background function: %v\n", err)
            }
        }()

        fn()
    }()
}

We can now use these two together to send OTPs to users asynchronously. But when authenticating users, we also need to send tokens. Let's create a new table in our database to store the tokens for each user.

Creating the Token Table in Database

 migrate create -ext=.sql -seq -dir=./migrations create-token

Add the follwing code to the 000002_create-token.up and 000002_create-token.down file:

-- 000002_create-token.up.sql

CREATE TABLE IF NOT EXISTS tokens (
    hash bytea PRIMARY KEY,
    user_id bigint NOT NULL REFERENCES users ON DELETE CASCADE,
    expiry timestamp(0) with time zone NOT NULL,
    scope text NOT NULL
);

-- 000002_create-token.down.sql
DROP  TABLE  IF  EXISTS tokens;

Explanation: The tokens table will store token data for each user, including the hashed token, associated user ID, expiry time, and scope.

Migrating the Database

migrate -path=./migrations -database="postgres://user:mysecretpassword@localhost:5432/cheershare?sslmode=disable" up

Token Model and Functions

Now, we’ll create the necessary functions to handle token generation, insertion, and retrieval. In the internals/data/models.go file, add the following code:

package data

import (
    "context"
    "crypto/rand"
    "crypto/sha256"
    "database/sql"
    "encoding/base32"
    "time"
)

const (
    ScopeAuthentication = "authentication"
)

type Token struct {
    Plaintext string    `json:"plaintext"`
    Hash      []byte    `json:"-"`
    UserId    int64     `json:"-"`
    Expiry    time.Time `json:"expiry"`
    Scope     string    `json:"-"`
}

type TokenModel struct {
    DB *sql.DB
}

/*
Function Parameters:
  - userId (int64): The unique identifier for the user for whom the token is being generated.
  - ttl (time.Duration): The time-to-live for the token, defining its expiry duration.
  - scope (string): The scope or purpose of the token (e.g., authentication, API access).

Return Values:
  - (*Token): A pointer to the generated Token structure, which contains:
  - UserId: The ID of the user associated with the token.
  - Expiry: The exact timestamp when the token will expire.
  - Scope: The defined purpose or scope of the token.
  - Plaintext: The plain, readable token string (used before hashing).
  - Hash: The SHA-256 hash of the plain token for secure storage/verification.
  - (error): An error object if token generation fails at any point.
*/
func generateToken(userId int64, ttl time.Duration, scope string) (*Token, error) {
    token := &Token{
        UserId: userId,
        Expiry: time.Now().Add(ttl),
        Scope:  scope,
    }

    randomBytes := make([]byte, 16)

    _, err := rand.Read(randomBytes)
    if err != nil {
        return nil, err
    }

    token.Plaintext = base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(randomBytes)
    hash := sha256.Sum256([]byte(token.Plaintext))
    token.Hash = hash[:]

    return token, nil

}

func (m TokenModel) Insert(token *Token) error {
    query := `INSERT INTO tokens (hash, user_id, expiry, scope)
    VALUES ($1, $2, $3, $4)`

    args := []interface{}{token.Hash, token.UserId, token.Expiry, token.Scope}

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    _, err := m.DB.ExecContext(ctx, query, args...)
    return err
}

func (m TokenModel) DeleteAllForUser(scope string, userID int64) error {
    query := `DELETE FROM tokens  WHERE user_id = $1 AND scope = $2`

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    _, err := m.DB.ExecContext(ctx, query, userID, scope)
    return err
}

func (m TokenModel) New(userId int64, ttl time.Duration, scope string) (*Token, error) {
    token, err := generateToken(userId, ttl, scope)
    if err != nil {
        return nil, err
    }

    err = m.Insert(token)
    return token, err
}

Explanation: This code defines the TokenModel struct and methods for generating and storing authentication tokens. The New function generates a new token, hashes it, and inserts it into the tokens table.

Don't forget to add this model to the models repository we defined earlier.

Updating the `signup` Handler

Only thing we need to do now if when we have verified otp, issue a new token and send it to user, New function will take care of inserting it into the db.

Let's also update the signup handler, right now we are using to different routes to send and verify otp, but when a user is already registred we don't really need to send an error, instead we can check if a user is already presend and based on thwt we can either follow the login flow or signup flow. open the cmd/api/user.go and add the following code.

/*
generateOTP generates a 4-digit OTP using a secure random number generator.
The function creates a random byte array and uses modulus operation to generate
a number between 0 and 9999. The result is formatted into a zero-padded string
to ensure it is always 4 digits long.
*/
func generateOTP() string {
    otp := make([]byte, 2)

    _, err := rand.Read(otp)
    if err != nil {
        log.Fatal("Error generating OTP:", err)
    }
    return fmt.Sprintf("%04d", int(otp[0])%10000)
}

/*
storeOTPInRedis stores user data, including the OTP, into Redis.
It uses the phone number as the key and stores the data as a hash.
A timeout context is created to prevent blocking indefinitely, and
the key is set to expire after 5 minutes to ensure security.
*/
func (app *application) storeOTPInRedis(ctx context.Context, phoneNumber, name, otp string) error {
    userData := map[string]string{
        "name": name,
        "otp":  otp,
    }

    err := app.cache.HSet(ctx, phoneNumber, userData).Err()
    if err != nil {
        return fmt.Errorf("failed to store user data in Redis: %w", err)
    }

    err = app.cache.Expire(ctx, phoneNumber, 5*time.Minute).Err()
    if err != nil {
        return fmt.Errorf("failed to set expiration for Redis key: %w", err)
    }

    return nil
}

/*
verifyOTPInRedis retrieves user data from Redis and validates the provided OTP.
It ensures that the OTP matches the value stored in Redis for the given phone number.
If the OTP is invalid or the key does not exist, an error is returned.
*/
func (app *application) verifyOTPInRedis(ctx context.Context, phoneNumber, otp string) (string, error) {
    userData, err := app.cache.HGetAll(ctx, phoneNumber).Result()
    if err != nil || len(userData) == 0 {
        return "", fmt.Errorf("invalid or expired OTP")
    }

    storedOTP := userData["otp"]
    if otp != storedOTP {
        return "", fmt.Errorf("invalid OTP")
    }

    return userData["name"], nil
}

/*
createUserIfNotExists checks if a user already exists in the database for the provided phone number.
If the user exists, their record is returned. Otherwise, a new user is created with the given name
and phone number. Any errors during database operations are propagated back to the caller.
*/
func (app *application) createUserIfNotExists(phoneNumber, name string) (*data.User, error) {
    user, err := app.models.User.GetByPhoneNumber(phoneNumber)
    if err == nil && user != nil {
        return user, nil
    }

    newUser := data.User{
        Name:        name,
        PhoneNumber: phoneNumber,
    }

    err = app.models.User.Insert(&newUser)
    if err != nil {
        return nil, fmt.Errorf("failed to create user: %w", err)
    }

    return &newUser, nil
}

/*
generateTokenForUser creates a new authentication token for the given user ID.
The token is valid for 48 hours and is associated with the "authentication" scope.
If token generation fails, an error is returned to the caller.
*/
func (app *application) generateTokenForUser(userID int64) (string, error) {
    token, err := app.models.Token.New(userID, 48*time.Hour, data.ScopeAuthentication)
    if err != nil {
        return "", fmt.Errorf("failed to generate token: %w", err)
    }

    return token.Plaintext, nil
}

/*
handleUserSignupAndVerification manages both user signup and OTP verification.
It combines the following functionality:
1. If OTP is not provided, it generates a new OTP and sends it to the user.
2. If OTP is provided, it validates the OTP and registers or retrieves the user.
3. Once the user is verified, an authentication token is created and sent as a response.

The process includes:
 1. Parsing and validating input data.
 2. Interacting with Redis for OTP storage and retrieval.
 3. Ensuring user data is either created or retrieved from the database.
 4. Generating an authentication token for successful verification.
 5. Returning appropriate error responses for any issues encountered.
*/
func (app *application) handleUserSignupAndVerification(w http.ResponseWriter, r *http.Request) {
    var input struct {
        Name        string `json:"name"`
        PhoneNumber string `json:"phone_number"`
        OTP         string `json:"otp"`
    }

    /*
        Read JSON from the request body.
        If an error occurs while reading, respond with a bad request error and log the issue.
    */
    err := app.readJSON(w, r, &input)
    if err != nil {
        app.errorResponse(w, http.StatusBadRequest, "Invalid request payload")
        app.logger.Println("Error reading JSON:", err)
        return
    }

    /*
        Validate required fields:
        - Phone number is always required.
        - Name is required only for OTP generation.
    */
    if input.PhoneNumber == "" {
        app.errorResponse(w, http.StatusBadRequest, "Phone number is required")
        return
    }

    if input.OTP == "" {
        /*
            OTP is not provided; generate a new OTP.
            1. Validate that the name is provided (required for OTP generation).
            2. Generate a new OTP.
            3. Store the OTP and name in Redis using the phone number as the key.
            4. Set an expiration time of 5 minutes for the Redis entry.
            5. Return a success response indicating the OTP was sent.
        */
        if input.Name == "" {
            app.errorResponse(w, http.StatusBadRequest, "Name is required for OTP generation")
            return
        }

        otp := generateOTP()

        ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
        defer cancel()

        err = app.storeOTPInRedis(ctx, input.PhoneNumber, input.Name, otp)
        if err != nil {
            app.errorResponse(w, http.StatusInternalServerError, "Failed to store OTP")
            app.logger.Println("Error storing OTP in Redis:", err)
            return
        }

        app.background(func() {
            err := app.sendOTPViaTwilio(otp, input.PhoneNumber)
            if err != nil {
                app.logger.Println("Error sending OTP via Twilio:", err)
            }
        })

        app.writeJSON(w, http.StatusOK, envelope{"success": true, "message": "OTP sent successfully"}, nil)
        return
    }

    /*
        OTP is provided; verify the OTP and proceed with user registration.
        1. Retrieve the stored OTP and user data from Redis using the phone number as the key.
        2. Validate the provided OTP against the stored OTP.
        3. If valid:
            a. Create or fetch the user in the database.
            b. Generate an authentication token for the user.
            c. Return a success response with user details and the token.
        4. If invalid, respond with an error indicating the OTP is invalid or expired.
    */
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    userName, err := app.verifyOTPInRedis(ctx, input.PhoneNumber, input.OTP)
    if err != nil {
        app.errorResponse(w, http.StatusUnauthorized, "Invalid or expired OTP")
        app.logger.Println("OTP verification failed for", input.PhoneNumber, ":", err)
        return
    }

    user, err := app.createUserIfNotExists(input.PhoneNumber, userName)
    if err != nil {
        app.errorResponse(w, http.StatusInternalServerError, "Failed to register user")
        app.logger.Println("Error registering user:", err)
        return
    }

    token, err := app.generateTokenForUser(user.ID)
    if err != nil {
        app.errorResponse(w, http.StatusInternalServerError, "Failed to generate authentication token")
        app.logger.Println("Error generating token for user ID", user.ID, ":", err)
        return
    }

    app.writeJSON(w, http.StatusOK, envelope{
        "success": true,
        "data":    user,
        "message": "User registered successfully",
        "token":   token,
    }, nil)
}

1. Introducing the Middleware Layers

To start, we’ll set up three key middlewares that ensure our app handles requests securely and consistently:

`recoverPanic` Middleware

The first middleware we’ll implement is the recoverPanic middleware. It’s a safety net for your application, catching any panics during request processing. If something goes wrong and causes a panic, it ensures the server doesn’t crash and provides a meaningful error response.

Explanation:

The defer block ensures that even if something breaks, we catch it, log it, and return a 500 Internal Server Error. We use this middleware to protect all the routes, so the application doesn’t crash in case of unforeseen errors.

`authenticate` Middleware

Next up is the authenticate middleware, which checks if the request has a valid Bearer token in the Authorization header. If the token is valid, we fetch the associated user from the database and attach them to the request’s context for further use.

Explanation:

The middleware checks if the Authorization header contains a Bearer token. It validates the token, retrieves the user associated with it, and attaches the user data to the request context. If the token is invalid, we return a 401 Unauthorized response.

`requireAuthenticatedUser` Middleware

For routes that require authentication, we use the requireAuthenticatedUser middleware to ensure the user is authenticated before allowing access.

Explanation:

This middleware checks the user attached to the request. If the user is anonymous (i.e., not authenticated), the middleware returns a 401 Unauthorized error. If the user is authenticated, it proceeds to the next handler.

package main

import (
    "errors"
    "net/http"
    "strings"

    "github.com/vishaaxl/cheershare/internal/data"
)

func (app *application) recoverPanic(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        defer func() {
            if err := recover(); err != nil {
                w.Header().Set("Connection", "close")
                app.errorResponse(w, http.StatusInternalServerError, "Failed to recover")
            }
        }()

        next.ServeHTTP(w, r)
    })
}

func (app *application) authenticate(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Add the "Vary: Authorization" header to the response. This indicates to any
        // caches that the response may vary based on the value of the Authorization
        // header in the request.
        w.Header().Add("Vary", "Authorization")

        // Retrieve the value of the Authorization header from the request. This will
        // return the empty string "" if there is no such header found.
        authorizationHeader := r.Header.Get("Authorization")
        // If there is no Authorization header found, use the contextSetUser() helper
        // that we just made to add the AnonymousUser to the request context. Then we
        // call the next handler in the chain and return without executing any of the
        // code below.
        if authorizationHeader == "" {
            r = app.contextSetUser(r, data.AnonymousUser)
            next.ServeHTTP(w, r)
            return
        }

        headerParts := strings.Split(authorizationHeader, " ")
        if len(headerParts) != 2 || headerParts[0] != "Bearer" {
            app.errorResponse(w, http.StatusUnauthorized, "Invalid authorization header")
            return
        }

        token := headerParts[1]
        if len(token) != 26 {
            app.errorResponse(w, http.StatusUnauthorized, "Invalid authorization header")
            return
        }
        // validate the token for length and required params

        user, err := app.models.User.GetForToken(data.ScopeAuthentication, token)

        if err != nil {
            switch {
            case errors.Is(err, data.ErrRecordNotFound):
                app.errorResponse(w, http.StatusUnauthorized, "Invalid authorization header")
            default:
                app.errorResponse(w, http.StatusInternalServerError, "Can't find user for specified token")
            }
            return
        }

        r = app.contextSetUser(r, user)
        next.ServeHTTP(w, r)
    })
}

func (app *application) requireAuthenticatedUser(next http.HandlerFunc) http.HandlerFunc {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

        user := app.contextGetUser(r)

        if user.IsAnonymous() {
            app.errorResponse(w, http.StatusUnauthorized, "Unauthorized")
            return
        }

        next.ServeHTTP(w, r)
    })
}

2. Context Management: Storing and Retrieving User Data

Now, let's dive into how we handle user data throughout the application. We use Go’s context package to attach user information to each request.

Create a new file in context.gpo in cmd/api directory and add the following code.

package main

import (
    "context"
    "net/http"

    "github.com/vishaaxl/cheershare/internal/data"
)

// contextKey is a custom type to avoid conflicts with other context keys in the application.
type contextKey string

// userContextKey is the key used to store and retrieve user information from the context.
const userContextKey = contextKey("cheershare.user")

// contextSetUser associates a given user object with the request's context.
//
// Parameters:
// - r: The incoming HTTP request.
// - user: A pointer to a data.User object that represents the authenticated user.
//
// Returns:
// - *http.Request: A new HTTP request with the user data stored in the context.
//
// Usage:
// This function is typically used after authenticating a user to attach the user information
// to the request, enabling downstream handlers to access the user data.
func (app *application) contextSetUser(r *http.Request, user *data.User) *http.Request {
    // Add the user to the context of the incoming request.
    ctx := context.WithValue(r.Context(), userContextKey, user)
    return r.WithContext(ctx)
}

// contextGetUser retrieves the user object from the request's context.
//
// Parameters:
// - r: The incoming HTTP request.
//
// Returns:
// - *data.User: A pointer to the user object stored in the context.
//
// Panics:
// If the user data is not present in the context or cannot be cast to *data.User,
// the function will panic with the message "missing user context."
//
// Usage:
// This function is used to access the user data attached to the request's context.
// It is generally called in handlers that need user-specific information.
func (app *application) contextGetUser(r *http.Request) *data.User {
    // Retrieve the user from the request context.
    user, ok := r.Context().Value(userContextKey).(*data.User)
    if !ok {
        panic("missing user context")
    }
    return user
}

3. Server Configuration: Putting It All Together

only thinng left for us to do is to use these middlewares, if you have any protected routes u can use requireAuthetincayted like:

router.HandlerFunc(http.MethodGet, "/protected", app.requireAuthenticatedUser(func(w http.ResponseWriter, r *http.Request) {
    err := app.writeJSON(w, http.StatusOK, envelope{"data": "Protected"}, nil)
    if err != nil {
        app.logger.Println(err)
        w.WriteHeader(500)
    }
}))

The recoverPanic and authenticate middlewares are applied globally to all routes.

/*
      Server configuration:
      - Address: Uses the configured port from `config`.
      - Handler: Routes handled by `httprouter`.
      - Timeouts: Configured for idle, read, and write operations.
      - Error logging: Logs errors during server startup or operation.
*/
srv := &http.Server{
    Addr:         fmt.Sprintf(":%d", app.config.port),
    Handler:      app.recoverPanic(app.authenticate(router)),
    IdleTimeout:  time.Minute,
    ReadTimeout:  10 * time.Second,
    WriteTimeout: 30 * time.Second,
}

4. Next Steps: Enhancements and Features

In the next sections, we will enhance this authentication system by adding file uploading, graceful shutdown for the server, and integrating metrics for better monitoring.

Here’s what’s coming next:

File Uploading: Learn how to implement file uploads and handle large files efficiently.
Metrics: Collect and monitor vital metrics for your application, such as request counts, response times, and error rates.
Graceful Shutdown: Ensure that your server shuts down gracefully, allowing for active connections to complete.

You can find the complete code for this tutorial on GitHub: Cheershare GitHub.

Part 1: https://dev.to/vishaaxl/build-an-otp-based-authentication-server-with-go-part-1-760
Part 2: https://dev.to/vishaaxl/build-an-otp-based-authentication-server-with-go-part-2-54gn

I’ve done my best to provide the most effective code and explanations, but if you think there’s anything I’ve overlooked or missed, please feel free to comment.

Contemplating 2024, Planning 2025

Vishal Shukla — Tue, 07 Jan 2025 13:59:33 +0000

2024 felt like a juggling act. I was learning Golang, Python, Docker, Kubernetes, networking, operating systems, DSA, and AI—all at once. The result? I made progress in many areas but didn’t feel confident about any of them.

Reflecting on this scattered approach, I realized that less is better. My plan for 2025 is to focus on a single topic each month and document my learning process and the projects I create.

🎯 Setting Better Goals

⏳ Parkinson's Law

Parkinson’s Law states that work expands to fill the time allotted for its completion. I’ve found this to be very true in my case—I tend to procrastinate and stretch tasks unnecessarily. To counter this, I plan to set tighter deadlines and stick to them, creating a sense of urgency to stay productive.

✍️ Writing

I’ve wanted to start writing for a long time, and now I’m finally committing to it. My goal is to write about my learning journey through blogs. I also want to be more mindful of how I spend my time and work toward using it effectively.

📆 Monthly Focus Areas

In 2025, I’ll dedicate each month to mastering one topic, whether it’s Golang, Python, Docker, Kubernetes, networking, operating systems, AI, or DSA. This focused approach will help me avoid spreading myself too thin and give me the time to go deeper into each area. By documenting my learning and projects, I’ll build a solid foundation and confidence in my skills.

🚀 One goal, one month at a time. Let’s do this!

Build an OTP-Based Authentication Server with Go: Part 2

Vishal Shukla — Mon, 06 Jan 2025 15:11:23 +0000

In this part, we will focus on the application's core functionality, such as sending and verifying OTPs. If you haven't already, I recommend reading Part 1 before continuing.

Setting Up the Database

To store users, we will use a PostgreSQL database. We'll define the necessary migrations using the migrate tool. Follow these steps to set up the database:

Create the Migration File In the root folder of your project, run the following command to create a migration for the users table:

    migrate create -ext=.sql -seq -dir=./migrations create-user-table

Explanation:

-ext=.sql: Specifies that the migration files should be created with the .sql extension.
-seq: Ensures the migration files are created with a sequential numbering system.
-dir=./migrations: Specifies the directory where migration files will be stored.

This command generates two files in the ./migrations directory:

000001_create-user-table.up.sql (for creating the table)
000001_create-user-table.down.sql (for rolling back the changes).

Define the User Schema

Open the 000001_create-user-table.up.sql file and add the following SQL commands:

-- 000001_create-user-table.up.sql

CREATE TABLE IF NOT EXISTS users (
    id bigserial PRIMARY KEY,
    created_at timestamp(0) with time zone NOT NULL DEFAULT NOW(),
    name text NOT NULL,
    phone_number text UNIQUE NOT NULL,
    version integer NOT NULL DEFAULT 1
);

Explanation:

id: A unique identifier for each user, auto-incremented using bigserial.
created_at: A timestamp recording when the user was created. The default value is the current timestamp.
- name: The user's name, stored as text.
- phone_number: The user's phone number, stored as unique text. This ensures no two users have the same phone number.
- version: An integer column, initialized to 1, to manage optimistic concurrency control if needed in future updates.

In the 000001_create-user-table.down.sql file, add the following SQL command to drop the table if needed:

-- 000001_create-user-table.down.sql

DROP TABLE IF EXISTS users;

Apply the Migration

To apply the migration to your PostgreSQL database, run the following command:

migrate -path=./migrations -database="postgres://user:mysecretpassword@localhost:5432/cheershare?sslmode=disable" up

Explanation:

-path=./migrations: Specifies the path where migration files are located.
-database: Specifies the connection string for the PostgreSQL database. Replace the values as needed:
user: Your database username (e.g., postgres).
password: Your database password (e.g., mysecretpassword).
dbname: The name of your database (e.g., cheershare).
sslmode=disable: Disables SSL for local development.

Adding Helper Functions for User Operations

Next, we’ll define helper functions to insert a user into the database and fetch a user by their phone number. These functions will allow us to interact with the users table efficiently.

1. Define `User` Struct and Helper Methods

Create a file named internal/data/user.go and add the following code:

package data

import (
    "context"
    "database/sql"
    "time"
)

type User struct {
    ID          int64     `json:"id"`
    CreatedAt   time.Time `json:"created_at"`
    Name        string    `json:"name"`
    PhoneNumber string    `json:"phone_number"`
    Version     int       `json:"version"`
}

type UserModel struct {
    DB *sql.DB
}

func (m UserModel) Insert(user *User) error {
    query := `
        INSERT INTO users (name, phone_number)
        VALUES ($1, $2)
        RETURNING id, created_at, version
    `

    args := []interface{}{user.Name, user.PhoneNumber}

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    err := m.DB.QueryRowContext(ctx, query, args...).Scan(&user.ID, &user.CreatedAt, &user.Version)
    if err != nil {
        return err
    }

    return nil
}

func (m UserModel) GetByPhoneNumber(PhoneNumber string) (*User, error) {
    query := `
        SELECT id, created_at, name, phone_number, version
        FROM users
        WHERE phone_number = $1
    `

    var user User

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*3)
    defer cancel()

    err := m.DB.QueryRowContext(ctx, query, PhoneNumber).Scan(&user.ID, &user.CreatedAt, &user.Name, &user.PhoneNumber, &user.Version)

    if err != nil {
        return nil, err
    }

    return &user, nil
}

Explanation:

Struct User: Represents a single user in the database. The JSON tags allow the struct to be serialized to JSON when needed.
Insert Method:
- Adds a new user to the database.
- Uses RETURNING to fetch the id, created_at, and version of the inserted user.
- A context with a 3-second timeout ensures the query does not hang indefinitely.
GetByPhoneNumber Method:
- Fetches a user by their phone number.
- Uses a SQL query with a WHERE clause to ensure the correct user is retrieved.
- Scans the result into a User struct.

2. Define a Central Repository for Models

Create a file named internal/data/models.go to define a central repository for all database models:

package data

import "database/sql"

type Models struct {
    User UserModel
}

func NewModels(db *sql.DB) Models {
    return Models{
        User: UserModel{
            DB: db,
        },
    }
}

Explanation:

Models Struct: Groups all models into a single struct. This design pattern simplifies access to database operations and improves maintainability.
NewModels Function: Instantiates the Models struct and assigns the database connection to each model. You can expand this to include other models as needed.

With these changes, you now have:

A UserModel for interacting with the users table.
Helper methods to insert and retrieve users by phone number.
A central repository (Models) to manage all database models efficiently.

1. Update the `application` Struct

The application struct in main.go is updated to include:

models: A central repository for database models, initialized with data.NewModels(db).

Additionally, we define routes for the new functionalities:

/signup: Handles user signup and OTP generation.
/verify: Verifies the OTP and registers the user.

Updated snippet from main.go:

// main.go

app := &application{
        config: *cfg,
        logger: logger,
        cache:  redisClient,
        models: data.NewModels(db),
}

router  :=  httprouter.New()
router.HandlerFunc(http.MethodPost, "/signup", app.signupUserHandler)
router.HandlerFunc(http.MethodPost, "/verify", app.verifyAndRegisterUserHandler)

2. OTP Generation and Signup Logic

Method: `signupUserHandler`

Purpose: Handles user signup and sends an OTP.
Steps:
1. Parse the JSON request body to extract the name and phone_number.
2. Validate the input and check if the phone number is already registered.
3. Generate a random 4-digit OTP using crypto/rand.
4. Store the OTP and name in Redis with the phone number as the key, setting a 5-minute expiration.
5. Log the OTP for debugging (in production, the OTP would be sent via SMS or email).
6. Respond with a success message.


func (app *application) signupUserHandler(w http.ResponseWriter, r *http.Request) {
    var input struct {
        Name        string `json:"name"`
        PhoneNumber string `json:"phone_number"`
    }

    /*
        Read JSON from request body.
        If an error occurs while reading, respond with a bad request error and log the issue.
    */
    err := app.readJSON(w, r, &input)
    if err != nil {
        app.errorResponse(w, http.StatusBadRequest, "Invalid request payload")
        app.logger.Println("Error reading JSON:", err)
        return
    }

    /*
        Ensure name and phone number are provided in the request.
        If any of them is missing, respond with a bad request error.
    */
    if input.Name == "" || input.PhoneNumber == "" {
        app.errorResponse(w, http.StatusBadRequest, "Name and phone number are required")
        return
    }

    /*
        Check if the user with provided phone number is already registered
    */
    user, err := app.models.User.GetByPhoneNumber(input.PhoneNumber)
    if err == nil && user != nil {
        app.errorResponse(w, http.StatusConflict, "User already exists with the given phone number")
        return
    }

    otp := generateOTP()

    /*
        Create a map to store the user data (name and OTP) in Redis.
        We are using the phone number as the key.
    */
    userData := map[string]string{
        "name": input.Name,
        "otp":  otp,
    }

    /*
        Set a timeout for Redis operations to prevent blocking indefinitely.
        If Redis operations fail, respond with an internal server error and log the issue.
    */
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    // Store user data in Redis using a hash
    err = app.cache.HSet(ctx, input.PhoneNumber, userData).Err()
    if err != nil {
        app.errorResponse(w, http.StatusInternalServerError, "Failed to store user data")
        app.logger.Println("Error storing user data in Redis:", err)
        return
    }

    /*
        Set an expiration time of 5 minutes for the Redis entry.
        This ensures the OTP will expire after 5 minutes for security purposes.
    */
    err = app.cache.Expire(ctx, input.PhoneNumber, time.Minute*5).Err()
    if err != nil {
        app.errorResponse(w, http.StatusInternalServerError, "Failed to set expiration for user data")
        app.logger.Println("Error setting expiration for Redis key:", err)
        return
    }

    // Log the OTP generation
    app.logger.Println("Generated OTP for", input.PhoneNumber, ":", otp)

    /*
        Respond with a success message indicating the OTP has been sent successfully.
        The response includes a JSON object with a success flag and a message.
    */
    app.writeJSON(w, http.StatusOK, envelope{"success": true, "message": "OTP sent successfully"}, nil)
}

3. OTP Verification and User Registration

Method: `verifyAndRegisterUserHandler`

Purpose: Verifies the OTP and registers the user in the database.
Steps:
1. Parse the JSON request body to extract the otp and phone_number.
2. Retrieve the stored OTP and user data from Redis.
3. Compare the provided OTP with the stored OTP.
4. If valid, insert the user into the database using UserModel.Insert.
5. Respond with a success message and user details.

func (app *application) verifyAndRegisterUserHandler(w http.ResponseWriter, r *http.Request) {
    var input struct {
        OTP         string `json:"otp"`
        PhoneNumber string `json:"phone_number"`
    }

    /*
        Read JSON from request body.
        If an error occurs while reading, respond with a bad request error.
    */
    err := app.readJSON(w, r, &input)
    if err != nil {
        app.errorResponse(w, http.StatusBadRequest, "Invalid request payload")
        return
    }

    /*
        Ensure OTP and phone number are provided in the request.
        If either is missing, respond with a bad request error.
    */
    if input.OTP == "" || input.PhoneNumber == "" {
        app.errorResponse(w, http.StatusBadRequest, "OTP and phone number are required")
        return
    }

    /*
        Set a timeout for Redis operations to prevent blocking indefinitely.
        If Redis operations fail, respond with an unauthorized error indicating invalid or expired OTP.
    */
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    // Retrieve user data from Redis using the phone number as the key
    userData, err := app.cache.HGetAll(ctx, input.PhoneNumber).Result()
    if err != nil || len(userData) == 0 {
        app.errorResponse(w, http.StatusUnauthorized, "Invalid or expired OTP")
        return
    }

    // Compare the provided OTP with the stored one in Redis
    storedOTP := userData["otp"]
    if input.OTP != storedOTP {
        app.errorResponse(w, http.StatusUnauthorized, "Invalid OTP")
        return
    }

    // OTP is valid, retrieve the user's name from Redis
    userName := userData["name"]

    /*
        Proceed with creating the user in the database or application.
        Example: app.createUser(userName, input.PhoneNumber)
    */
    user := data.User{
        Name:        userName,
        PhoneNumber: input.PhoneNumber,
    }

    err = app.models.User.Insert(&user)

    if err != nil {
        app.errorResponse(w, http.StatusInternalServerError, "Failed to register user")
        app.logger.Println("Error registering user:", err)
        return
    }

    /*
        Respond with a success message indicating the user has been registered successfully.
        The response includes a JSON object with a success flag and a message containing the user's name.
    */
    app.writeJSON(w, http.StatusOK, envelope{
        "success": true,
        "data":    user,
        "message": "User registered successfully",
    }, nil)
}

Key Points:

Redis ensures the OTP is valid for a limited time, preventing replay attacks.
The database insert operation is performed only after successful OTP verification.

Here's an explanation of the additions and how the provided code integrates into the overall OTP-based authentication server:

4. Utility Function: `generateOTP`

A helper function to generate a secure 4-digit OTP:

func generateOTP() string {
    otp := make([]byte, 2)

    _, err := rand.Read(otp)
    if err != nil {
        log.Fatal("Error generating OTP:", err)
    }
    return fmt.Sprintf("%04d", int(otp[0])%10000)
}

Explanation:

Uses crypto/rand for secure random number generation.
Converts the byte value to a 4-digit number.

5. Response Handling

The signupUserHandler and verifyAndRegisterUserHandler use helper functions like app.readJSON, app.writeJSON, and app.errorResponse to handle:

JSON parsing and serialization.
Error responses with appropriate HTTP status codes.
Success responses with user-friendly messages.

Lets implement those as well, create a new helper.go file in the api folder and paste the following code.

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
)

type envelope map[string]interface{}

func (app *application) errorResponse(w http.ResponseWriter, status int, message interface{}) {
    env := envelope{"error": message}

    err := app.writeJSON(w, status, env, nil)
    if err != nil {
        app.logger.Println(err)
        w.WriteHeader(500)
    }

}

func (app *application) writeJSON(w http.ResponseWriter, status int, body envelope, headers http.Header) error {
    js, err := json.Marshal(body)
    if err != nil {
        return err
    }

    js = append(js, '\n')

    for key, value := range headers {
        w.Header()[key] = value
    }

    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    w.Write(js)

    return nil
}

func (app *application) readJSON(w http.ResponseWriter, r *http.Request, dst interface{}) error {
    maxBytes := 104856
    r.Body = http.MaxBytesReader(w, r.Body, int64(maxBytes))
    // Decode the request body into the target destination.

    // Initialize the json.Decoder, and call the DisallowUnknownFields() method on it
    // before decoding. This means that if the JSON from the client now includes any
    // field which cannot be mapped to the target destination, the decoder will return
    // an error instead of just ignoring the field.
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields()

    // Decode the request body to the destination.
    err := dec.Decode(dst)
    if err != nil {
        // If there is an error during decoding, start the triage...
        var syntaxError *json.SyntaxError
        var unmarshalTypeError *json.UnmarshalTypeError
        var invalidUnmarshalError *json.InvalidUnmarshalError
        switch {
        // Use the errors.As() function to check whether the error has the type
        // *json.SyntaxError. If it does, then return a plain-english error message
        // which includes the location of the problem.
        case errors.As(err, &syntaxError):
            return fmt.Errorf("body contains badly-formed JSON (at character %d)", syntaxError.Offset)
        // In some circumstances Decode() may also return an io.ErrUnexpectedEOF error
        // for syntax errors in the JSON. So we check for this using errors.Is() and
        // return a generic error message. There is an open issue regarding this at
        // https://github.com/golang/go/issues/25956.
        case errors.Is(err, io.ErrUnexpectedEOF):
            return errors.New("body contains badly-formed JSON")
        // Likewise, catch any *json.UnmarshalTypeError errors. These occur when the
        // JSON value is the wrong type for the target destination. If the error relates
        // to a specific field, then we include that in our error message to make it
        // easier for the client to debug.
        case errors.As(err, &unmarshalTypeError):
            if unmarshalTypeError.Field != "" {
                return fmt.Errorf("body contains incorrect JSON type for field %q", unmarshalTypeError.Field)
            }
            return fmt.Errorf("body contains incorrect JSON type (at character %d)", unmarshalTypeError.Offset)
        // An io.EOF error will be returned by Decode() if the request body is empty. We
        // check for this with errors.Is() and return a plain-english error message
        // instead.
        case errors.Is(err, io.EOF):
            return errors.New("body must not be empty")
        // A json.InvalidUnmarshalError error will be returned if we pass a non-nil
        // pointer to Decode(). We catch this and panic, rather than returning an error
        // to our handler. At the end of this chapter we'll talk about panicking
        // versus returning errors, and discuss why it's an appropriate thing to do in
        // this specific situation.
        case errors.As(err, &invalidUnmarshalError):
            panic(err)
        // For anything else, return the error message as-is.
        default:
            return err
        }
    }

    // Call Decode() again, using a pointer to an empty anonymous struct as the
    // destination. If the request body only contained a single JSON value this will
    // return an io.EOF error. So if we get anything else, we know that there is
    // additional data in the request body and we return our own custom error message.
    err = dec.Decode(&struct{}{})

    if err != io.EOF {
        return errors.New("body must only contain a single JSON value")
    }

    return nil
}

Testing the Application

With the code in place, you’re ready to test the application. Follow these steps:

Start the Application:

Run the following command to start your application in the background using Docker Compose:

docker-compose up -d

Run the Server:

Once the containers are up and running, you can start your Go server by executing

go run ./cmd/api

Test the Endpoints:

Now that the server is running, you can test the /signup and /verify endpoints to ensure everything works as expected.

Next Steps: Implementing Authentication

Once the basic signup and OTP verification flow is working, the next step is to secure the application using authenticated routes. Here's how you can proceed:

1.Token-Based Authentication:

Implement JWT (JSON Web Tokens) for secure, stateless authentication.
After verifying the OTP and registering the user, generate a JWT that will be sent back to the client.
The client can use this token in the Authorization header for subsequent requests to access protected resources.

2.Middleware:

Create middleware to validate JWT tokens in the request headers.
Only allow authenticated users to access certain routes.

3.Integrate Twilio:

Use Twilio for sending OTP messages instead of generating the OTP manually.
Integrate the Twilio API to send SMS with the OTP to the user’s phone number.

Part 3: https://dev.to/vishaaxl/build-an-otp-based-authentication-server-with-go-part-3-2jc0

Build an OTP-Based Authentication Server with Go: Part 1

Vishal Shukla — Sat, 04 Jan 2025 19:42:43 +0000

Getting Started

Begin by creating a new folder for your project and initialize a Go module with the following command:

go mod init github.com/vishaaxl/cheershare

Set up the Project Structure

Start by setting up a new Go project with the following folder structure:

my-otp-auth-server/
├── cmd/
│   └── api/
│       └── main.go
│       └── user.go
│       └── token.go
├── internal/
│   └── data/
│       ├── models.go
│       └── user.go
│       └── token.go
├── docker-compose.yml
├── go.mod
└── Makefile

Next, set up your docker-compose.yml file. This configuration will define the services—PostgreSQL and Redis—that you'll be working with throughout this tutorial.

Setting Up Services with Docker Compose

We will start by configuring the services required for our project. For the backend, we need the following:

Redis: We'll use the redis:6 image. This service will configure a password for secure access, expose port 6379, and enforce password authentication using the --requirepass flag to secure Redis access.
PostgreSQL: We'll use the postgres:13 image. The service will define a default user, password, and database, expose port 5432 for communication, and persist data with a named volume (postgres_data) to ensure durability.

Optional:

Main Backend Service: You can define the main backend service here as well, which will interact with both PostgreSQL and Redis.

// docker-compose.yml
services:
  postgres:
    image: postgres:13
    container_name: postgres
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: mysecretpassword
      POSTGRES_DB: cheershare
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data

  redis:
    image: redis:6
    container_name: redis
    environment:
      REDIS_PASSWORD: mysecretpassword
    ports:
      - "6379:6379"
    command: ["redis-server", "--requirepass", "mysecretpassword"]

volumes:
  postgres_data:

Main Backend Service

For routing and handling HTTP requests, we’ll use the github.com/julienschmidt/httprouter package. To install the dependency, run the following command:

go get github.com/julienschmidt/httprouter

Next, create a file at cmd/api/main.go and paste the following code. An explanation for each line is provided in the comments:

//  main.go
package main

import (
    "fmt"
    "log"
    "net/http"
    "os"
    "time"

    "github.com/julienschmidt/httprouter"
)

/*
config struct:
- Holds application-wide configuration settings such as:
  - `port`: The port number on which the server will listen.
  - `env`: The current environment (e.g., "development", "production").
*/
type config struct {
    port int
    env  string
}

/*
application struct:
- Encapsulates the application's dependencies, including:
  - `config`: The application's configuration settings.
  - `logger`: A logger instance to handle log messages.
*/
type application struct {
    config config
    logger *log.Logger
}

func main() {
    cfg := &config{
        port: 4000,
        env:  "development",
    }

    logger := log.New(os.Stdout, "INFO\t", log.Ldate|log.Ltime)

    app := &application{
        config: *cfg,
        logger: logger,
    }

    router := httprouter.New()

    router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
        w.WriteHeader(http.StatusOK)
        fmt.Fprintln(w, "Welcome to the Go application!")
    })

    /*
        Initialize the HTTP server
        - Set the server's address to listen on the specified port.
        - Assign the router as the handler.
        - Configure timeouts for idle, read, and write operations.
        - Set up an error logger to capture server errors.
    */
    srv := &http.Server{
        Addr:         fmt.Sprintf(":%d", app.config.port),
        Handler:      router,
        IdleTimeout:  time.Minute,
        ReadTimeout:  10 * time.Second,
        WriteTimeout: 30 * time.Second,
    }

    app.logger.Printf("Starting server on port %d in %s mode", app.config.port, app.config.env)

    err := srv.ListenAndServe()
    if err != nil {
        app.logger.Fatalf("Could not start server: %s", err)
    }
}

Right now, you can test your setup by starting out server using go run ./cmd/api and sending a request to http://localhost:4000, which will return a welcome message. Next, we’ll define three additional routes to implement our core functionality:

/send-otp: This route will handle sending OTPs to users. It will generate a unique OTP, store it in Redis, and deliver it to the user.
/verify-otp: This route will verify the OTP provided by the user. It will check against the value stored in Redis to confirm the user's identity.
/login: This route will handle user login functionality once the OTP is verified and user is successfully created.

But before we continue, we need a way to store user information like phone number and their one-time password for which we need to connect to the services we defined earlier in the docker-compose.yml file.

Defining Helper Functions

Before implementing the routes, let’s define two essential helper functions. These functions will handle connections to the Redis and PostgreSQL servers, ensuring that our backend can interact with these services.

Modify the 'config' struct to store information about the services. These functions are pretty self-explanatory.

//main.go

// connectDB establishes a connection to the PostgreSQL database using the provided configuration.
// It returns a database connection instance or an error if the connection fails.
func connectDB(cfg config) (*sql.DB, error) {
    db, err := sql.Open("postgres", cfg.db.dsn)
    if err != nil {
        return nil, err
    }

    // Set the maximum number of open connections and idle connections for the database pool.
    db.SetMaxOpenConns(cfg.db.maxOpenConns)
    db.SetMaxIdleConns(cfg.db.maxIdleConns)
    db.SetConnMaxIdleTime(cfg.db.maxIdleTime)

    // Create a context with a timeout of 5 seconds to ensure the connection attempt doesn't hang indefinitely.
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    // Ping the database to verify the connection is successful.
    err = db.PingContext(ctx)
    if err != nil {
        return nil, err
    }

    // Return the connected database instance if successful.
    return db, nil
}

// connectRedis establishes a connection to the Redis server using the provided configuration.
// It returns a Redis client instance or an error if the connection fails.
func connectRedis(cfg redisConfig) (*redis.Client, error) {
    client := redis.NewClient(&redis.Options{
        Addr:     cfg.addr,     // Redis server address
        Password: cfg.password, // Redis server password
        DB:       cfg.db,       // Redis database index
    })

    // Create a context with a timeout of 5 seconds for the Redis connection.
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    // Ping the Redis server to verify the connection.
    _, err := client.Ping(ctx).Result()
    if err != nil {
        return nil, fmt.Errorf("unable to connect to Redis: %w", err)
    }

    // Return the connected Redis client instance if successful.
    return client, nil
}

You can use these functions to establish a connection to the PostgreSQL database and Redis server after starting the services with the docker-compose up -d command.

But first lets install the required dependencies.

go get github.com/lib/pq
go get github.com/go-redis/redis/v8

In the next part, we'll start working on those routes we talked about earlier. This is what your main.go file should look like now.

//  main.go
package main

import (
    "context"
    "database/sql"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"

    "github.com/go-redis/redis/v8"
    "github.com/julienschmidt/httprouter"
    _ "github.com/lib/pq"
)

/*
config struct:
- Holds application-wide configuration settings such as:
  - `port`: The port number on which the server will listen.
  - `env`: The current environment (e.g., "development", "production").
  - `db`: Database-specific configurations.
  - `redis`: Redis-specific configurations.
*/
type config struct {
    port  int
    env   string
    db    db
    redis redisConfig
}

type db struct {
    dsn          string
    maxOpenConns int
    maxIdleConns int
    maxIdleTime  time.Duration
}

type redisConfig struct {
    addr     string
    password string
    db       int
}

/*
application struct:
- Encapsulates the application's dependencies, including:
  - `config`: The application's configuration settings.
  - `logger`: A logger instance to handle log messages.
  - `redis`: A Redis client instance for caching.
*/
type application struct {
    config config
    logger *log.Logger
    cache  *redis.Client
}

func main() {
    /*
       Configuration includes:
       - `port`: The port on which the server will run (default is 4000).
       - `env`: The environment mode (e.g., "development", "production").
       - `db`: Database connection settings, including the DSN, connection limits, and idle timeout.
       - `redis`: Redis connection settings, including server address, password, and database index.
    */
    cfg := &config{
        port: 4000,
        env:  "development",
        db: db{
            dsn:          "user=postgres password=mysecretpassword dbname=cheershare sslmode=disable",
            maxOpenConns: 25,
            maxIdleConns: 25,
            maxIdleTime:  time.Minute,
        },
        redis: redisConfig{
            addr:     "localhost:6379",
            password: "mysecretpassword",
            db:       0,
        },
    }

    /*
       Logger settings:
       - Prefix: "INFO\t" indicates informational logs.
       - Flags: Includes date and time for log entries.
    */
    logger := log.New(os.Stdout, "INFO\t", log.Ldate|log.Ltime)

    /*
       - connectDB establishes a connection using the database configuration.
       - If the connection fails, the application logs a fatal error and exits.
       - If successful, logs a success message and defers closing the connection.
    */
    db, err := connectDB(cfg.db)
    if err != nil {
        logger.Fatalf("Failed to connect to database: %s", err)
    }
    logger.Println("Connected to PostgreSQL database")
    defer db.Close()

    /*
       - connectRedis establishes a connection using the Redis configuration.
       - If the connection fails, the application logs a fatal error and exits.
       - If successful, logs a success message and defers closing the connection.
    */
    redisClient, err := connectRedis(cfg.redis)
    if err != nil {
        logger.Fatalf("Failed to connect to Redis: %s", err)
    }
    logger.Println("Connected to Redis server")
    defer redisClient.Close()

    app := &application{
        config: *cfg,
        logger: logger,
    }

    router := httprouter.New()
    router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
        w.WriteHeader(http.StatusOK)
        fmt.Fprintln(w, "Welcome to the Go application!")
    })

    /*
       Server configuration:
       - Address: Uses the configured port from `config`.
       - Handler: Routes handled by `httprouter`.
       - Timeouts: Configured for idle, read, and write operations.
       - Error logging: Logs errors during server startup or operation.
    */
    srv := &http.Server{
        Addr:         fmt.Sprintf(":%d", app.config.port),
        Handler:      router,
        IdleTimeout:  time.Minute,
        ReadTimeout:  10 * time.Second,
        WriteTimeout: 30 * time.Second,
    }

    // Log server start information
    app.logger.Printf("Starting server on port %d in %s mode", app.config.port, app.config.env)

    // Start the server and handle potential errors
    err = srv.ListenAndServe()
    if err != nil {
        app.logger.Fatalf("Could not start server: %s", err)
    }
}

func connectDB(cfg db) (*sql.DB, error) {
    db, err := sql.Open("postgres", cfg.dsn)

    if err != nil {
        return nil, err
    }

    db.SetMaxOpenConns(cfg.maxOpenConns)
    db.SetMaxIdleConns(cfg.maxIdleConns)
    db.SetConnMaxIdleTime(cfg.maxIdleTime)

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    err = db.PingContext(ctx)
    if err != nil {
        return nil, err
    }

    return db, nil
}

func connectRedis(cfg redisConfig) (*redis.Client, error) {
    client := redis.NewClient(&redis.Options{
        Addr:     cfg.addr,
        Password: cfg.password,
        DB:       cfg.db,
    })

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()

    _, err := client.Ping(ctx).Result()
    if err != nil {
        return nil, err
    }

    return client, nil
}

Part 2: https://dev.to/vishaaxl/build-an-otp-based-authentication-server-with-go-part-2-54gn

Beginner Python Project: Build an Augmented Reality Drawing App Using OpenCV and Mediapipe

Vishal Shukla — Tue, 31 Dec 2024 19:48:39 +0000

In this Python project, we'll create a simple AR drawing app. Using your webcam and hand gestures, you can draw virtually on the screen, customize your brush, and even save your creations!

Setup

To get started, create a new folder and initialize a new virtual environment using:

python -m venv venv

./venv/Scripts/activate

Next up install the required libraries using pip or installer of your choice:

pip install mediapipe

pip install opencv-python

Note

You may have trouble installing mediapipe with latest version on python. As I am writing this blog I am using python 3.11.2. Make sure to use the compatible version on python.

Step 1: Capture Webcam Feed

The first step is to set up your webcam and display the video feed. We'll use OpenCV's VideoCapture to access the camera and continuously display frames.

import cv2  

# The argument '0' specifies the default camera (usually the built-in webcam).
cap = cv2.VideoCapture(0)

# Start an infinite loop to continuously capture video frames from the webcam
while True:
    # Read a single frame from the webcam
    # `ret` is a boolean indicating success; `frame` is the captured frame.
    ret, frame = cap.read()

    # Check if the frame was successfully captured
    # If not, break the loop and stop the video capture process.
    if not ret:
        break

    # Flip the frame horizontally (like a mirror image)
    frame = cv2.flip(frame, 1)

    # Display the current frame in a window named 'Webcam Feed'
    cv2.imshow('Webcam Feed', frame)

    # Wait for a key press for 1 millisecond
    # If the 'q' key is pressed, break the loop to stop the video feed.
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the webcam resource to make it available for other programs
cap.release()

# Close all OpenCV-created windows
cv2.destroyAllWindows()

Did You Know?

When using cv2.waitKey() in OpenCV, the returned key code may include extra bits depending on the platform. To ensure you correctly detect key presses, you can mask the result with 0xFF to isolate the lower 8 bits (the actual ASCII value). Without this, your key comparisons might fail on some systems—so always use & 0xFF for consistent behavior!

Step 2: Integrate Hand Detection

Using Mediapipe's Hands solution, we'll detect the hand and extract the position of key landmarks like the index and middle fingers.

import cv2  
import mediapipe as mp

# Initialize the MediaPipe Hands module
mp_hands = mp.solutions.hands  # Load the hand-tracking solution from MediaPipe
hands = mp_hands.Hands(
    min_detection_confidence=0.9,
    min_tracking_confidence=0.9 
)

cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
    if not ret:
        break 

    # Flip the frame horizontally to create a mirror effect
    frame = cv2.flip(frame, 1)

    # Convert the frame from BGR (OpenCV default) to RGB (MediaPipe requirement)
    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    # Process the RGB frame to detect and track hands
    result = hands.process(frame_rgb)

    # If hands are detected in the frame
    if result.multi_hand_landmarks:
        # Iterate through all detected hands
        for hand_landmarks in result.multi_hand_landmarks:
            # Get the frame dimensions (height and width)
            h, w, _ = frame.shape

            # Calculate the pixel coordinates of the tip of the index finger
            cx, cy = int(hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].x * w), \
                     int(hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].y * h)

            # Calculate the pixel coordinates of the tip of the middle finger
            mx, my = int(hand_landmarks.landmark[mp_hands.HandLandmark.MIDDLE_FINGER_TIP].x * w), \
                     int(hand_landmarks.landmark[mp_hands.HandLandmark.MIDDLE_FINGER_TIP].y * h)

            # Draw a circle at the index finger tip on the original frame
            cv2.circle(frame, (cx, cy), 10, (0, 255, 0), -1)  # Green circle with radius 10

    # Display the processed frame in a window named 'Webcam Feed'
    cv2.imshow('Webcam Feed', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break  # Exit the loop if 'q' is pressed

# Release the webcam resources for other programs
cap.release()
cv2.destroyAllWindows()

Step 3: Track Finger Position and Draw

We’ll track the index finger and allow drawing only when the index and middle fingers are separated by a threshold distance.

We'll maintain a list of co-ordinates of the index fingers to draw on the original frame and every time middle finger is close enough, we'll append None to this co-ordinates array indicating a breakage.

import cv2  
import mediapipe as mp  
import math  

# Initialize the MediaPipe Hands module
mp_hands = mp.solutions.hands
hands = mp_hands.Hands(
    min_detection_confidence=0.9,  
    min_tracking_confidence=0.9   
)

# Variables to store drawing points and reset state
draw_points = []  # A list to store points where lines should be drawn
reset_drawing = False  # Flag to indicate when the drawing should reset

# Brush settings
brush_color = (0, 0, 255)  
brush_size = 5 


cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()  
    if not ret:
        break 

    frame = cv2.flip(frame, 1) 
    frame_rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) 
    result = hands.process(frame_rgb)  

    # If hands are detected
    if result.multi_hand_landmarks:
        for hand_landmarks in result.multi_hand_landmarks:
            h, w, _ = frame.shape  # Get the frame dimensions (height and width)

            # Get the coordinates of the index finger tip
            cx, cy = int(hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].x * w), \
                     int(hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP].y * h)

            # Get the coordinates of the middle finger tip
            mx, my = int(hand_landmarks.landmark[mp_hands.HandLandmark.MIDDLE_FINGER_TIP].x * w), \
                     int(hand_landmarks.landmark[mp_hands.HandLandmark.MIDDLE_FINGER_TIP].y * h)

            # Calculate the distance between the index and middle finger tips
            distance = math.sqrt((mx - cx) ** 2 + (my - cy) ** 2)

            # Threshold distance to determine if the fingers are close (used to reset drawing)
            threshold = 40 

            # If the fingers are far apart
            if distance > threshold:
                if reset_drawing:  # Check if the drawing was previously reset
                    draw_points.append(None)  # None means no line
                    reset_drawing = False  
                draw_points.append((cx, cy))  # Add the current point to the list for drawing
            else:  # If the fingers are close together set the flag to reset drawing
                reset_drawing = True  # 

    # Draw the lines between points in the `draw_points` list
    for i in range(1, len(draw_points)):
        if draw_points[i - 1] and draw_points[i]:  # Only draw if both points are valid
            cv2.line(frame, draw_points[i - 1], draw_points[i], brush_color, brush_size)


    cv2.imshow('Webcam Feed', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the webcam and close all OpenCV windows
cap.release()
cv2.destroyAllWindows()

Step 4: Improvements

Use OpenCV rectangle() and putText() for buttons to toggle brush size and color.
Add an option to save the frame.
Add an eraser tool, use the new co-ordinates to modify draw_points array.

First blog!

Vishal Shukla — Sun, 29 Dec 2024 18:46:36 +0000

Beginner Go Project: Create a Task Runner in Go

Vishal Shukla ・ Dec 29 '24

#webdev #go #beginners #programming

Beginner Go Project: Create a Task Runner in Go

Vishal Shukla — Sun, 29 Dec 2024 18:42:32 +0000

What we are going to build

We'll be making a tool like make which we can use run tasks using a simple yaml file like this.

tasks:
    build:
        description:  "compile the project"
        command:  "go build main.go"
        dependencies:  [test]
    test:
        description:  "run unit tests"
        command:  "go test -v ./..."

Lets get started, first we need to outline the course of action. We've already defined the task file schema. We can use json instead of yaml but for the sake of this project we are going to go with yml files.

From the file we can see that we will need a struct to store a single task and a way to run dependent tasks before going forward with the main one. Let's start with initiating our project. Create a new folder and run:

go mod init github.com/vishaaxl/mommy

You can name your project however you want, I'm going with this 'mommy' name. We also need to install some package to work with yaml files - basically converting them into a map object. Go ahead and install the following package.

go get gopkg.in/yaml.v3

Next up create a new main.go file and start with defining the 'Task' struct.

package main

import (
    "gopkg.in/yaml.v3"
)
// Task defines the structure of a task in the configuration file.
// Each task has a description, a command to run, and a list of dependencies
// (other tasks that need to be completed before this task).
type Task struct {
    Description  string   `yaml:"description"`  // A brief description of the task.
    Command      string   `yaml:"command"`      // The shell command to execute for the task.
    Dependencies []string `yaml:"dependencies"` // List of tasks that need to be completed before this task.
}

This one is pretty self explantory. This will hold the value of each individual task. Nextup we need one more struct to store list of tasks and load the contents of the .yaml file into this new object.

// Config represents the entire configuration file,
// which contains a map of tasks by name.
type Config struct {
    Tasks map[string]Task `yaml:"tasks"` // A map of task names to task details.
}

// loadConfig reads and parses the configuration file (e.g., Makefile.yaml),
// and returns a Config struct containing the tasks and their details.
func loadConfig(filename string) (Config, error) {
    // Read the content of the config file.
    data, err := os.ReadFile(filename)
    if err != nil {
        return Config{}, err
    }

    // Unmarshal the YAML data into a Config struct.
    var config Config
    err = yaml.Unmarshal(data, &config)
    if err != nil {
        return Config{}, err
    }

    return config, nil
}

Next up we need to create a function that executes a single task. We'll be using os/exec module to run the task in the shell. In Golang, the os/exec package provides a way to execute shell commands and external programs.

// executeTask recursively executes the specified task and its dependencies.
// It first ensures that all dependencies are executed before running the current task's command.
func executeTask(taskName string, tasks map[string]Task, executed map[string]bool) error {
    // If the task has already been executed, skip it.
    if executed[taskName] {
        return nil
    }

    // Get the task details from the tasks map.
    task, exists := tasks[taskName]
    if !exists {
        return fmt.Errorf("task %s not found", taskName)
    }

    // First, execute all the dependencies of this task.
    for _, dep := range task.Dependencies {
        // Recursively execute each dependency.
        if err := executeTask(dep, tasks, executed); err != nil {
            return err
        }
    }

    // Now that dependencies are executed, run the task's command.
    fmt.Printf("Running task: %s\n", taskName)
    fmt.Printf("Command: %s\n", task.Command)

    // Execute the task's command using the shell (sh -c allows for complex shell commands).
    cmd := exec.Command("sh", "-c", task.Command)
    cmd.Stdout = os.Stdout // Direct standard output to the terminal.
    cmd.Stderr = os.Stderr // Direct error output to the terminal.

    // Run the command and check for any errors.
    if err := cmd.Run(); err != nil {
        return fmt.Errorf("failed to execute command %s: %v", task.Command, err)
    }

    // Mark the task as executed.
    executed[taskName] = true
    return nil
}

Now we have all the building blocks of the program we can use them in the main function to load the config file and start automating. We are going to use flag package to read the command line flags.

func main() {
    // Define command-line flags
    configFile := flag.String("f", "Mommy.yaml", "Path to the configuration file") // Path to the config file (defaults to Makefile.yaml)
    taskName := flag.String("task", "", "Task to execute")                             // The task to execute (required flag)

    // Parse the flags
    flag.Parse()

    // Check if the task flag is provided
    if *taskName == "" {
        fmt.Println("Error: Please specify a task using -task flag.")
        os.Exit(1) // Exit if no task is provided
    }

    // Load the configuration file
    config, err := loadConfig(*configFile)
    if err != nil {
        fmt.Printf("Failed to load config: %v\n", err)
        os.Exit(1) // Exit if the configuration file can't be loaded
    }

    // Map to track which tasks have been executed already (avoiding re-execution).
    executed := make(map[string]bool)

    // Start executing the specified task (with dependencies)
    if err := executeTask(*taskName, config.Tasks, executed); err != nil {
        fmt.Printf("Error executing task: %v\n", err)
        os.Exit(1) // Exit if task execution fails
    }
}

Let's test the whole thing out, create a new Mommy.yaml and paste the yaml code from the start into it. we will use out task runner to create binaries for our project. Run:

go run main.go -task build

If everything goes fine, you'll see a new .exe file in the root of the folder. Great, we have a working task runner now. We can add the location of this .exe file in the environment variables of our system and use this from anywhere using:

 mommy -task build

Complete Code

package main

import (
    "flag"
    "fmt"
    "os"
    "os/exec"
    "gopkg.in/yaml.v3"
)

// Task defines the structure of a task in the configuration file.
// Each task has a description, a command to run, and a list of dependencies
// (other tasks that need to be completed before this task).
type Task struct {
    Description  string   `yaml:"description"`  // A brief description of the task.
    Command      string   `yaml:"command"`      // The shell command to execute for the task.
    Dependencies []string `yaml:"dependencies"` // List of tasks that need to be completed before this task.
}

// Config represents the entire configuration file,
// which contains a map of tasks by name.
type Config struct {
    Tasks map[string]Task `yaml:"tasks"` // A map of task names to task details.
}

// loadConfig reads and parses the configuration file (e.g., Makefile.yaml),
// and returns a Config struct containing the tasks and their details.
func loadConfig(filename string) (Config, error) {
    // Read the content of the config file.
    data, err := os.ReadFile(filename)
    if err != nil {
        return Config{}, err
    }

    // Unmarshal the YAML data into a Config struct.
    var config Config
    err = yaml.Unmarshal(data, &config)
    if err != nil {
        return Config{}, err
    }

    return config, nil
}

// executeTask recursively executes the specified task and its dependencies.
// It first ensures that all dependencies are executed before running the current task's command.
func executeTask(taskName string, tasks map[string]Task, executed map[string]bool) error {
    // If the task has already been executed, skip it.
    if executed[taskName] {
        return nil
    }

    // Get the task details from the tasks map.
    task, exists := tasks[taskName]
    if !exists {
        return fmt.Errorf("task %s not found", taskName)
    }

    // First, execute all the dependencies of this task.
    for _, dep := range task.Dependencies {
        // Recursively execute each dependency.
        if err := executeTask(dep, tasks, executed); err != nil {
            return err
        }
    }

    // Now that dependencies are executed, run the task's command.
    fmt.Printf("Running task: %s\n", taskName)
    fmt.Printf("Command: %s\n", task.Command)

    // Execute the task's command using the shell (sh -c allows for complex shell commands).
    cmd := exec.Command("sh", "-c", task.Command)
    cmd.Stdout = os.Stdout // Direct standard output to the terminal.
    cmd.Stderr = os.Stderr // Direct error output to the terminal.

    // Run the command and check for any errors.
    if err := cmd.Run(); err != nil {
        return fmt.Errorf("failed to execute command %s: %v", task.Command, err)
    }

    // Mark the task as executed.
    executed[taskName] = true
    return nil
}

func main() {
    // Define command-line flags
    configFile := flag.String("f", "Makefile.yaml", "Path to the configuration file") // Path to the config file (defaults to Makefile.yaml)
    taskName := flag.String("task", "", "Task to execute")                             // The task to execute (required flag)

    // Parse the flags
    flag.Parse()

    // Check if the task flag is provided
    if *taskName == "" {
        fmt.Println("Error: Please specify a task using -task flag.")
        os.Exit(1) // Exit if no task is provided
    }

    // Load the configuration file
    config, err := loadConfig(*configFile)
    if err != nil {
        fmt.Printf("Failed to load config: %v\n", err)
        os.Exit(1) // Exit if the configuration file can't be loaded
    }

    // Map to track which tasks have been executed already (avoiding re-execution).
    executed := make(map[string]bool)

    // Start executing the specified task (with dependencies)
    if err := executeTask(*taskName, config.Tasks, executed); err != nil {
        fmt.Printf("Error executing task: %v\n", err)
        os.Exit(1) // Exit if task execution fails
    }
}

DEV Community: Vishal Shukla

Intermediate Go Tutorial: Building Your Own Redis

Prerequisites

Folder Structure

Implementation

cache.go: Implementing the LRU Cache

persistence.go: Managing Persistence for the Cache

main.go: Starting the Redis-like Server

Learn Python by Building a Lisp Interpreter

Introduction to Lisp

Example of Lisp Syntax

Steps to Implement a Lisp Interpreter

1. Tokenization:

2. Parsing:

3. Environment Setup:

Why Do We Need the Env Class?

4. Evaluation:

5. REPL (Read-Eval-Print Loop):

Conclusion

Build an OTP-Based Authentication Server with Go: Part 4 - File Uploads and Graceful Shutdown

Graceful Shutdown

Automating Commands with Makefile

Run a Make Command

Add More Commands

File upload and tracking

Controller for File Upload and Tracking

Building a String Calculator with Test-Driven Development (TDD): A Step-by-Step Guide

Getting started

Step 1: Empty String Should Return "0"

Expected Output:

Step 2: Adding One or Two Numbers Should Return Their Sum

Writing the Test

Implementing the Code

Step 3 : Handling Multiple Numbers

Writing the Test

Step 4: Handling New Lines Between Numbers

Writing the Test

Implementing the Code

Step 5: Handling Custom Delimiters

Writing the Test

Implementing the Code

Step 6: Handling Negative Numbers

Writing the Test

Implementing the Code

Step 7: Counting Add Method Calls

Writing the Test

Implementing the Code

Step 8 & 9: Ignore Numbers Greater Than 1000 and Handle Custom Delimiters of Any Length

Writing the Tests

Implementing the Code

Step 10: Multiple Delimiters Support

Writing the Test

Implementing the Code

Testing It

Expected Output

Build an OTP-Based Authentication Server with Go: Part 3

Sending OTP with Twilio

Improving Performance with Goroutines

Creating the Token Table in Database

Migrating the Database

Token Model and Functions

Updating the signup Handler

1. Introducing the Middleware Layers

recoverPanic Middleware

authenticate Middleware

requireAuthenticatedUser Middleware

2. Context Management: Storing and Retrieving User Data

3. Server Configuration: Putting It All Together

4. Next Steps: Enhancements and Features

Contemplating 2024, Planning 2025

🎯 Setting Better Goals

⏳ Parkinson's Law

✍️ Writing

📆 Monthly Focus Areas

Build an OTP-Based Authentication Server with Go: Part 2

Setting Up the Database

Adding Helper Functions for User Operations

1. Define User Struct and Helper Methods

2. Define a Central Repository for Models

1. Update the application Struct

`cache.go`: Implementing the LRU Cache

`persistence.go`: Managing Persistence for the Cache

`main.go`: Starting the Redis-like Server

Why Do We Need the `Env` Class?

Updating the `signup` Handler

`recoverPanic` Middleware

`authenticate` Middleware

`requireAuthenticatedUser` Middleware

1. Define `User` Struct and Helper Methods

1. Update the `application` Struct

Method: `signupUserHandler`

Method: `verifyAndRegisterUserHandler`

4. Utility Function: `generateOTP`