DEV Community: Sumit Negi

💡 Using the `log/slog` Package in Go for Effective Logging in Web Applications

Sumit Negi — Mon, 27 Jan 2025 15:22:00 +0000

💡 Using the `log/slog` Package in Go for Effective Logging in Web Applications

Logging is a crucial aspect of building robust and maintainable web applications. In Go, the log/slog package provides a simple, structured, and concurrency-safe way to manage logs. Whether you're debugging, monitoring, or troubleshooting, proper logging allows you to track the behavior and performance of your application. In this article, we'll explore how to use the log/slog package and explain its importance in a web application.

📊 Why Use `log/slog`?

The log/slog package in Go allows you to log structured data with various severity levels. This structured logging is particularly useful in modern applications, where logs are used not only for debugging but also for performance tracking and error monitoring. By using structured logs, you can include more meaningful information, such as request data, method names, or error details, which is harder to achieve with traditional text-based logs.

📝 Key Features of `log/slog`

📈 Structured Logging: log/slog supports logging with key-value pairs, making logs more readable and useful for debugging and monitoring.
🔒 Concurrency-Safety: Custom loggers created by log/slog are designed to be used across multiple goroutines without introducing race conditions, making them ideal for web applications where multiple requests are handled concurrently.
🔧 Custom Log Handlers: You can create custom handlers that output logs in different formats (like JSON or text) or direct them to different destinations, such as files or logging services.
🔄 Log Levels: The package provides several log levels (e.g., Debug, Info, Warn, Error), allowing you to categorize the importance of your log messages.

🛠️ Setting Up `log/slog` in a Go Web Application

Let’s walk through an example of using log/slog in a simple Go web application. This example will show how to set up a logger and use it to log HTTP requests, errors, and other information.

🔄 Step 1: Installing and Importing `log/slog`

To get started, first import the log/slog package:

import "log/slog"

🌟 Step 2: Create a Logger

We can create a logger by choosing a handler. In the example below, we use the JSONHandler, which formats logs as JSON.

logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
    Level: slog.LevelDebug,  // Set the minimum log level
    AddSource: true,         // Include the source file and line number in the log
}))

💪 Step 3: Log Requests and Information

Now that we have a logger, let’s use it to log some HTTP request details. Here’s a basic example of a Go web server using log/slog:

package main

import (
    "flag"
    "log/slog"
    "net/http"
    "os"
)

type application struct {
    logger *slog.Logger
}

func main() {
    mux := http.NewServeMux()

    addr := flag.String("addr", ":4000", "HTTP network address")
    logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
        Level: slog.LevelDebug,
        AddSource: true,
    }))
    flag.Parse()

    app := &application{
        logger: logger,
    }

    mux.HandleFunc("/", app.home)

    logger.Info("starting server", slog.String("addr", *addr))

    err := http.ListenAndServe(*addr, mux)
    if err != nil {
        logger.Error("server error", slog.String("error", err.Error()))
        os.Exit(1)
    }
}

func (app *application) home(w http.ResponseWriter, r *http.Request) {
    app.logger.Info("request received", slog.String("method", r.Method), slog.String("path", r.URL.Path))

    w.Write([]byte("Hello, World!"))
}

🔍 Why Structured Logging is Important in a Web App

🎨 Easier Debugging: With structured logs, you can log additional context such as request methods, paths, user IDs, and error details.
🌐 Improved Monitoring: Structured logs allow you to track user interactions, detect anomalies, and gather metrics for performance tuning.
🎉 Concurrency-Safety: log/slog ensures logs from multiple goroutines don’t interfere with each other.
🌯 Customizable Output: Easily adjust the log output format and destination.
💡 Maintainability: Structured logging helps maintain high-quality code as your application grows.

🔮 Advanced Logging Features

Log Levels: Categorize messages at different severity levels.
Log Rotation: Archive and rotate log files for efficient storage.
Remote Logging: Send logs to centralized systems like Grafana Loki, Datadog, or ELK stack.

🌟 Conclusion

Logging is essential for any application, but structured logging takes it to the next level. By using Go's log/slog package, you can generate logs that are easy to understand, filter, and analyze. In a web application, logging helps you monitor user requests, track errors, and improve the overall user experience. By following best practices and utilizing log/slog features, you'll build more robust and maintainable Go applications.

AWS EC2 ALB setup with ASG

Sumit Negi — Fri, 22 Mar 2024 09:53:49 +0000

Creating an Amazon Machine Image (AMI) from an EC2 instance and then using it to set up an Application Load Balancer (ALB), a target group, and an Auto Scaling Group (ASG) involves several steps. This guide will walk you through the process, leveraging both the AWS Management Console for a comprehensive understanding.

Creating an AMI from an EC2 Instance

Using the AWS Management Console:
- Navigate to the EC2 Dashboard and select "Instances" from the sidebar.
- Right-click the instance you wish to create an AMI from and choose "Create Image" from the context menu.
- In the "Create Image" dialog box, provide a unique name and description for your AMI.
- By default, AWS will shut down the instance, take snapshots of any attached volumes, create and register the AMI, and then reboot the instance. You can choose "No reboot" if you prefer not to shut down the instance, but be aware that this might affect the file system integrity of the created image.

Creating an Application Load Balancer (ALB)

Using the AWS Management Console:
- Go to the EC2 Dashboard and select "Load Balancers" under the "Load Balancing" section.
- Click "Create Load Balancer" and choose "Application Load Balancer".
- Follow the wizard to configure your ALB, including specifying the name, scheme (Internet-facing or Internal), and IP address type (IPv4 or Dualstack). You'll also need to select the VPC, subnets, and security groups for your ALB.

Creating a Target Group

Using the AWS Management Console:
- Navigate to the EC2 Dashboard and select "Target Groups" under the "Load Balancing" section.
- Click "Create target group" and specify the name, target type (e.g., instance, IP), protocol, and port. You'll also need to select the VPC for your target group.

Creating an Auto Scaling Group (ASG) with a Launch Template

Using the AWS Management Console:
- Go to the EC2 Dashboard and select "Auto Scaling Groups" under the "Auto Scaling" section.
- Click "Create Auto Scaling group" and follow the wizard to configure your ASG. You'll need to specify the launch template, which includes the AMI ID you created earlier.

Using AWS Certificate Manager (ACM) to Forward Traffic from HTTP to HTTPS

Request or Import an SSL/TLS Certificate in ACM:
- Request a Public Certificate: Navigate to the ACM console and request a public certificate for your domain. You'll need to validate ownership of the domain by adding a CNAME record to your DNS configuration or by using an email validation method.
- Import a Certificate: If you have an existing certificate from a third party, you can import it into ACM. This includes the certificate body, private key, and certificate chain (if applicable).
Create an HTTPS Listener for Your ALB:
- Go to the EC2 Dashboard and select "Load Balancers".
- Choose your ALB and navigate to the "Listeners" tab.
- Click "Add listener" and select "HTTPS" as the protocol.
- For the "Default SSL/TLS certificate", choose "From ACM" and select the certificate you requested or imported.
Configure Redirection from HTTP to HTTPS:
- Still in the "Listeners" tab, select the HTTP listener (if it exists) and choose "Edit".
- Change the "Default action(s)" to "Redirect to" and specify the HTTPS listener you created.
- You can choose between a temporary (HTTP 302) or permanent (HTTP 301) redirect based on your needs.
Associate the ACM Certificate with Your ALB:
- Ensure that the ACM certificate is associated with your ALB as the default SSL/TLS certificate for the HTTPS listener. This step is crucial for the ALB to terminate HTTPS connections using the certificate.
Update DNS Records (Optional):
- If you're using a custom domain, update your DNS records to point to your ALB. This step is necessary for clients to resolve your domain name to the ALB's IP address.

Additional Considerations

Security Policy: When configuring your HTTPS listener, it's recommended to use the latest predefined security policy to ensure the highest level of security.
Multiple Certificates: ALBs support multiple SSL/TLS certificates using Server Name Identification (SNI). This allows you to serve different certificates based on the domain name requested by the client.

By following these steps, you can secure your application by forwarding traffic from HTTP to HTTPS using ACM and configuring your ALB to use HTTPS. This setup not only enhances the security of your application but also simplifies the management of SSL/TLS certificates.

AWS EC2 Node.js deployment

Sumit Negi — Wed, 13 Mar 2024 18:38:09 +0000

Step 1: Launching an EC2 Instance

Go to EC2 and click on Launch Instance.
Add a name for your instance.
Select Ubuntu from Quick Start.
Choose the desired instance type.
For connecting to a Key pair, use a pre-existing key or create a new one.
In the Network setting, add a security group. A security group acts as a firewall, controlling both incoming and outgoing traffic from your instance.
Storage configuration is fine as gp2.

After this connect to your EC2 instance install node and create a mini project:

Installation guide:

https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-20-04

Create a new project:

npm init -y
add a new file index.js

const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

npm i pm2 express
pm2 start index.js
pm2 save

Caddy
Step 1: Install Caddy
https://caddyserver.com/docs/install

sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
sudo vim /etc/caddy/Caddyfile
:80 {
    reverse_proxy localhost:3000
}
sudo systemctl restart caddy

Update the Caddyfile to use your domain name and enable HTTPS.

sudo vim /etc/caddy/Caddyfile

mydomain.com {
    reverse_proxy localhost:3000
}
sudo systemctl restart caddy

Apache Kafka: A brief beginner's guide part 1

Sumit Negi — Tue, 13 Feb 2024 07:29:18 +0000

Kafka: An Open-Source Distributed Event Streaming Platform

Kafka is an open-source distributed event streaming platform that enables publishing, subscribing, storing, and processing streams of records in real-time. It serves as a highly scalable, fault-tolerant, and durable messaging system, commonly used for building real-time data pipelines, streaming analytics, and event-driven architectures.

Core Concepts of Kafka

At its core, Kafka is a distributed publish-subscribe messaging system. Data is written to Kafka topics by producers and consumed from those topics by consumers. Kafka topics can be partitioned, enabling parallel data processing, and can be replicated across multiple brokers for fault tolerance.

Understanding Events and Event Streaming in Kafka

Apache Kafka serves as a distributed event streaming platform used for building real-time data pipelines and streaming applications. Events and event streaming are fundamental concepts in Kafka:

Events

In Kafka, an event represents a piece of data, often in the form of a message, produced by a publisher (producer) and consumed by one or more subscribers (consumers). Events are typically small units of data, such as log entries or transactions. They are immutable, meaning once produced, they cannot be modified. Instead, new events can be produced to reflect updates or changes.

Event Streaming

Event streaming involves continuously capturing, processing, and distributing events in real-time. Kafka provides a distributed, fault-tolerant, and scalable infrastructure for event streaming. It allows applications to publish events to topics and consume events from topics asynchronously, facilitating various use cases such as real-time analytics, log aggregation, and messaging systems.

Key Concepts Related to Events and Event Streaming in Kafka

Brokers

Brokers are servers in the Kafka storage layer that store event streams from one or more sources. A Kafka cluster typically consists of several brokers. Each broker in a cluster serves as a bootstrap server, enabling connections to every broker in the cluster.

Producers

Producers are applications that generate events and publish them to Kafka topics. They determine the topics to which events will be sent and how events are partitioned among different partitions within a topic.

Consumers

Consumers retrieve events from Kafka topics. Each consumer maintains its position within a topic using metadata called the offset. Consumers can read events sequentially or move to specific offsets to reprocess past data.

Topics

Topics are categories or channels to which events are published. Each topic consists of partitions, allowing parallel processing and scalability. Events within topics are immutable and organized for efficient storage and retrieval.

Partitions

Partitions are segments of a topic where events are stored in an ordered sequence. Each partition is replicated across multiple Kafka brokers for fault tolerance.

Offsets

Offsets are unique identifiers assigned to each event within a partition. They track the position of consumers within a partition.

Consumer Groups

Consumer groups collectively consume events from one or more topics. Each consumer group maintains its offset for each partition it consumes from, enabling parallel event processing.

Replication

Replication ensures data redundancy and fault tolerance. Kafka replicates topics across brokers, ensuring multiple copies of data for resilience.

Installation Steps

Install Docker and Docker Compose.
git clone kafka docker repo
cd kafka-stack-docker-compose.
Run: docker-compose -f zk-single-kafka-single.yml up -d
Check to make sure both the services are running: docker-compose -f zk-single-kafka-single.yml ps

Node.js Coding

Create a kafkaClient.js file:

import { Kafka } from "kafkajs"

export const kafka = new Kafka({
  clientId: 'my-app',
  brokers: ['localhost:9092'],
})

This code initializes a Kafka client using the kafkajs library, specifying a client ID and broker address for connecting to a Kafka cluster.

producer.js:

import { kafka } from "./kafkaClient.js"
import { Partitioners } from "kafkajs"

const producer = kafka.producer({
  createPartitioner: Partitioners.LegacyPartitioner
})

await producer.connect()

setInterval(async ()=>{
  await producer.send({
    topic: 'test-topic',
    messages: [
      { value: `Hello, what's up ${new Date()}` },
    ],
  })
},3000)

This code imports a Kafka client from a module, sets up a producer with a legacy partitioner, connects it to Kafka, then sends messages to the 'test-topic' topic every 3 seconds with a timestamp.

consumer.js:

import { kafka } from "./kafkaClient.js"

const consumer = kafka.consumer({ groupId: 'test-group' })

await consumer.connect()
await consumer.subscribe({ topic: 'test-topic', fromBeginning: true })

await consumer.run({
  eachMessage: async ({ topic, partition, message }) => {
    console.log({
      value: message.value.toString(),
    })
  },
})

This code imports a Kafka client from a module, sets up a consumer with a specified group ID, connects it to Kafka, subscribes to the 'test-topic' topic from the beginning, then runs a loop to handle incoming messages, logging their values to the console.

Thank you for reading. Have a wonderful day!

DNS basics

Sumit Negi — Tue, 06 Feb 2024 07:25:31 +0000

What is DNS?

Domain Name System which translates the human friendly hostnames into the machine IP addresses

Each device connected to the Internet has a unique IP address which other machines use to find the device. DNS servers eliminate the need for humans to memorize IP addresses such as 192.168.1.1 (in IPv4), or more complex newer alphanumeric IP addresses such as 2400:cb00:2048:1::c629:d7a2 (in IPv6).

DNS Terminologies

• Domain Registrar : Amazon Route 53, GoDaddy,namecheap …
• DNS Records: A, AAAA, CNAME, NS, …
• Zone File: contains DNS records
• Name Server : resolves DNS queries (Authoritative or Non-Authoritative)
• Top Level Domain (TLD): .com, .us, .in, .gov, .org, …
• Second Level Domain (SLD): amazon.com, google.com

There are 4 DNS servers involved in loading a web page:

Here's a simplified explanation of the terms:

DNS recursor: It's like a librarian who helps you find a book in a library. When you ask for a webpage, your device talks to the DNS recursor, which then searches for the webpage's location.
Root nameserver: This is the first place the DNS recursor goes to find the address of a webpage. It's like the index in a library that shows where different sections of books are located.
TLD nameserver: It's like a specific section of books in a library. When the root nameserver points to a TLD nameserver, it's like finding a specific section of books related to a certain topic, such as ".com" websites.
Authoritative nameserver: Think of it as a dictionary on a shelf of books. It's the final place the DNS recursor checks to find the exact address of the webpage you're looking for. If it finds the address, it tells the DNS recursor, which then gives you the webpage's location.

In simpler terms, an authoritative DNS server and a recursive DNS resolver play different roles in the DNS system.

Recursive DNS resolver: This is like a detective that helps you find information. When you type a web address into your browser, the recursive resolver starts looking for the address. It asks other servers for help until it finds the right one. Sometimes, it already knows where to look because it remembers previous searches (caching).
Authoritative DNS server: This is like the ultimate source of information. It's the server that actually has the address you're looking for. When the recursive resolver finally finds the right server, that server gives it the exact address you need. This is what allows your browser to find and display the website you want to visit.

Cloudflare, for example, not only provides recursive DNS resolver services like Google DNS or OpenDNS but also hosts infrastructure-level authoritative nameservers, like the F-root server network. These authoritative nameservers are essential for handling vast amounts of DNS traffic and ensuring the smooth functioning of the Internet.

There are the simplified steps in a DNS lookup:

A user types a website name (like example.com) into a web browser.
The request travels to a DNS resolver.
The resolver asks a root nameserver for help.
The root nameserver points the resolver to a TLD (like .com).
The resolver asks the TLD server for the domain's nameserver.
The TLD server gives the resolver the nameserver's IP address.
The resolver queries the domain's nameserver.
The nameserver provides the IP address for the website.
The resolver sends the IP address back to the web browser.

These steps allow the web browser to find the correct IP address for the requested website.

After obtaining the IP address for example.com through the DNS lookup process, the browser proceeds with the following steps to fetch the webpage:

The browser sends a HTTP request to the obtained IP address.
The server located at that IP address responds by sending the webpage content back to the browser.
The browser then renders and displays the webpage for the user to interact with.

j

Sumit Negi — Wed, 24 Jan 2024 06:32:27 +0000

DEV Community: Sumit Negi

💡 Using the `log/slog` Package in Go for Effective Logging in Web Applications

💡 Using the log/slog Package in Go for Effective Logging in Web Applications

📊 Why Use log/slog?

📝 Key Features of log/slog

🛠️ Setting Up log/slog in a Go Web Application

🔄 Step 1: Installing and Importing log/slog

🌟 Step 2: Create a Logger

💪 Step 3: Log Requests and Information

🔍 Why Structured Logging is Important in a Web App

🔮 Advanced Logging Features

🌟 Conclusion

AWS EC2 ALB setup with ASG

Creating an AMI from an EC2 Instance

Creating an Application Load Balancer (ALB)

Creating a Target Group

Creating an Auto Scaling Group (ASG) with a Launch Template

Using AWS Certificate Manager (ACM) to Forward Traffic from HTTP to HTTPS

Additional Considerations

AWS EC2 Node.js deployment

Apache Kafka: A brief beginner's guide part 1

Kafka: An Open-Source Distributed Event Streaming Platform

Core Concepts of Kafka

Understanding Events and Event Streaming in Kafka

Events

Event Streaming

Key Concepts Related to Events and Event Streaming in Kafka

Brokers

Producers

Consumers

Topics

Partitions

Offsets

Consumer Groups

Replication

Installation Steps

Node.js Coding

DNS basics

What is DNS?

DNS Terminologies

j

💡 Using the `log/slog` Package in Go for Effective Logging in Web Applications

📊 Why Use `log/slog`?

📝 Key Features of `log/slog`

🛠️ Setting Up `log/slog` in a Go Web Application

🔄 Step 1: Installing and Importing `log/slog`