DEV Community: Ukeje Chukwuemeriwo Goodness

Maintainability Is All You Need

Ukeje Chukwuemeriwo Goodness — Thu, 09 Jan 2025 14:47:24 +0000

Great technical documentation is easy to update and upgrade to suit all project stakeholders. The ideal technical documentation strikes the fine line between being comprehensive enough to cover all necessary details and concise enough to remain straightforward to understand.

Over time, your documentation may not strike the right notes. You may build more features, or developers could, and you’ll need to refactor your project’s documentation. So, you have to factor in maintainability during the documentation engineering process.

Understanding Maintainability for Technical Documentation

Mainability is the metric for how easy it is to keep documentation accurate, relevant, and up-to-date. Maintainable documentation is structured, consistent, and modular. Incorporating changes should be as easy as editing any document for any stakeholder.

Maintaining your product documentation will require extra effort and time, but it’s worth it if you’re playing the long game to onboard more developers than your competitors; you’d agree that your docs are failing if developers still need to ask further questions. Improving your documentation’s maintainability could fix that!

You’ll save time for all stakeholders since your documentation is easy to fix when there are issues. This reduces the cost of reinventing your documents, and ultimately, everyone is happy because there are:

Developers can update the documentation to help other developers with similar issues.
Any duplicate questions will hardly be directed to your team.
Your documentation is a perpetual motion machine that doesn’t need much to maintain.

These perks are easy to achieve, but you’ll need to be intentional from the start, from choosing a tool to shipping the documentation.

Implementation Strategies for Maintainable Documentation

Maintainability is a process of improving the overall status. Here are some strategies you can implement to make your documents more maintainable.

Docs as Code

Docs as Code is the blue pill if you’re considering long-term document maintenance, especially for engineering teams.

Treating your documentation like any other part of your codebase with version control systems like Git to track changes across the product would keep your product and documentation in sync.

Also, enforce code reviews for updates and integrate documentation updates into your CI/CD pipeline so your documentation evolves with your code.

Automate Testing and Validation

Manual validation of documentation is time-consuming and prone to errors. Automating these processes not only saves time but also improves accuracy.

Try out linting, grammar check and typography tools to enforce style and grammar consistency in your documentation, you can add one to your CICD processes too lint before deployment.

Content Reuse Frameworks

Duplication is the enemy of maintainability. Content reuse allows you to write information once and reuse it across multiple documentation pages or products. This strategy ensures consistency and reduces the overhead of updating the same content in various locations.

Create reusable content blocks for recurring information, such as installation instructions or API references. Structured reuse ensures consistency and saves time when updates are necessary.

Establish a Review and Update Process

Maintaining documentation means you’ll have to review it regularly to ensure that it stays relevant and you’re hitting the nail on the head with content, especially when working with cross-functional teams.

Steps to Build an Effective Review Process:

Define Ownership: Assign specific team members responsibility for different documentation sections.
Set Review Cadence: Schedule periodic reviews (e.g., quarterly or after major product releases) to identify outdated content.
Feedback Loops: Create channels for users and developers to report issues or suggest improvements in your documentation.
Version Updates: Align documentation updates with product releases, ensuring that new features and changes are accurately reflected.

Integrating this process into your development workflow ensures documentation becomes a natural part of your product lifecycle.

Keep All Stakeholders Involved

Maintainable documentation is a collaborative effort. Developers, product managers, technical writers, and other stakeholders should contribute to and maintain the documentation. This will create a more comprehensive and useful knowledge base involving diverse stakeholders.

You can keep all stakeholders involved by:

Use accessible tools like GitBook and Mintlify to build your documentation.
Use easy-to-understand markup languages like Markdown so anybody can propose changes with minimal overhead.
Host regular sync-ups between all stakeholders to discuss updates and pain points.
Train team members on how to contribute to the documentation effectively.

If they interact with your documentation, they’re stakeholders themselves, so try roping them into your processes.

Conclusion

You’ve learned the importance of maintainability and how it keeps your documentation relevant over time.

Maintainability is not just a feature of great documentation. It’s a critical investment in your project’s development and technical marketing. Remember, the key is to treat documentation with the same rigour and attention as your codebase while ensuring it remains accessible to all stakeholders.

Reasons Why Developers Hate Your Docs

Ukeje Chukwuemeriwo Goodness — Mon, 06 Jan 2025 12:24:09 +0000

Technical marketing wasn’t spun out of thin air. You’ll need to take your project’s documentation more seriously to grow your developer base, especially if you’ve built a developer-centric product. You can’t mid-curve this.

Here, you’ll learn the main pain points in your documentation for developers and how you can fix these issues to onboard more developers and grow your developer base.

The Documentation Deficit

Content is king; your documentation is a precious artefact, and I mean that in the most pragmatic way possible. Your documentation should cover all facets of the product and leave no stone unturned in answering any developer's questions about exploring and using the product.

Developers should be able to dive right in when they visit your documentation. It is common to see technical documentation that doesn’t cover all the facets of the project under the guise that developers join forums to ask questions, which is tedious. Personally, if you have competitors, they have better documentation. I’m bought, and this is the consensus for many developers.

Basic Technical Writing Problems

Technical documentation should be direct and concise, hitting the nail and helping developers build from the get-go.

When writing the content, your goal should be structured logical communication that developers of all skill levels associated with your documentation would understand without assuming preexisting knowledge.

Unclear objectives and insufficient examples are another turn-off; implementation should be straightforward.

Poor DX Features

DX (Developer Experience) is the critical factor that expresses how pleased developers are with your documentation, which is the objective of the documentation in the first place.

Features like search with relevant results for key terms or topics, code snippet copying and duplication, and accessibility features like keyboard navigation or screen reader support are non-negotiable for technical documentation.

Trading off any of these would significantly degrade your documentation’s DX and send them to your competitors.

Maintainability Factors

Maintainability refers to how easily documentation can be improved and modified over time. Modularity, versioning, structural consistency, automation, and stakeholder collaboration are crucial for maintaining documentation.

Developers themselves should be able to contribute to your docs if they solve any undocumented issues or choose to improve on the clarity of what you’ve documented.

Fixing These Documentation Issues

The most obvious way to fix these issues would be to hire a great technical writer. These issues are mostly due to development teams trying to underscore documentation by writing it themselves.

Document every part of your product that the users interact with; do not assume any parts would be prominent.
Get a documentation audit to see what and where you can improve.
Add surveys to your documentation to get feedback on how you can improve your documentation.
Pay attention to your onboarding process; you want to give developers an easy start on your product.
Use developer personas and add prerequisites to help developers of all skill levels.
Add UX-improving features like search, code snippet copying, etc. and implement navigation logically.
Add a troubleshooting section to help fix issues on the fly and a Glossary if you have jargon specific to your app.
Automate documentation processes with tools like Postman to autogenerate API implementation in any language. Invest in CI/CD integration to ensure code releases match the documentation and lining tools for grammar and style consistency.

Great documentation is progressive. You should also pay attention to maintainability. Add support for different versions and improve your docs-as-code process with CICD support for the technologies you’re using for your documentation.

Conclusion

You’ve learned about some popular reasons developers may hate your documentation and how you can fix them.

Collaborating with your developers primarily through self-service options, like well-written and designed documentation, is a great way to ensure the success of your product. Please don’t sleep on it.

Beginner's Guide to Managing Files in Amazon S3 with Go

Ukeje Chukwuemeriwo Goodness — Mon, 04 Nov 2024 12:51:21 +0000

Amazon Simple Storage Service (S3) has become the industry standard for web-based file storage. It is designed for up to 99.999999999% durability and security. It also has flexible storage classes, data management, and analytics features.

Are you building the server-side app for your project in Go and need a storage option? This article assumes that you’re an AWS beginner and walks you through the processes of using S3 for storage from your Go apps.

Getting Started with AWS and Go

Setting up an AWS S3 bucket with the right permissions and parameters can be daunting. You’ll have to create a bucket, create an IAM user, issue permissions to the IAM user to execute operations on the bucket, and set up the access keys in your environment.

First, create an Amazon AWS account and sign in if you don’t have one yet. Then, search for S3 like this.

Now, Click on “Create a bucket.” You’ll be prompted to configure the bucket according to your project’s specifications.

Now that you’ve created a bucket, the next step is to set up an IAM user to whom you’ll issue permission to access the S3 bucket.

Head to the security credentials section of your profile and create a user.

Then, create an access key for the user and retrieve the access and secret keys.

On your computer, in the root directory, create a .aws folder and a file named credentials with no extension; then add the keys like this:

[default]
aws_access_key_id = <aws_access_key_id here>
aws_secret_access_key = <aws_secret_access_key here>

Now, Issue these permissions to the user to allow S3 uploads.

Finally, initialize a Go project and install the AWS Go SDK.

go mod tidy
go get github.com/aws/aws-sdk-go

You’re all set up, and now, you can start uploading, downloading and managing files with AWS S3 Buckets.

Uploading Files to an S3 Bucket

First, you must import the necessary packages from the AWS SDK package.

Add these modules to the top of your main.go file or whichever file you’re using:

package main

import (
    "bytes"
    "fmt"
    "io"
    "os"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

To upload a file, you must create a new session, open a file, and use the session instance to upload the file.

func UploadFile(bucket, region, filePath, key, acl string) error {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String(region),
    })
    if err != nil {
        return fmt.Errorf("error creating session: %w", err)
    }
    file, err := os.Open(filePath)
    if err != nil {
        return fmt.Errorf("error opening file: %w", err)
    }
    defer file.Close()

    var buf bytes.Buffer
    if _, err := io.Copy(&buf, file); err != nil {
        return fmt.Errorf("error reading file: %w", err)
    }

    _, err = s3.New(sess).PutObject(&s3.PutObjectInput{
        Bucket: aws.String(bucket), 
        Key:    aws.String(key),          
        Body:   bytes.NewReader(buf.Bytes()), 
        ACL: aws.String(acl),     
    })
    if err != nil {
        return fmt.Errorf("error uploading file: %w", err)
    }

    fmt.Println("File uploaded successfully:", filePath, "to key:", key)
    return nil
}

The UploadFile function takes in the bucket name, AWS region, the file key and ACL (access controls list) and uploads the specified file to S3 with the specified key.

The session.NewSession function creates a new AWS session and s3.New creates a new S3 session. The PutObject function takes in a reference to the PutObjectInput with the file Bucket, Key, file body, and ACL.

func main() {
    bucket := "cloudboxbucket"  
    region := "eu-north-1"      
    filePath := "Makefile"     
    key := "Makefile"           
    acl := "private"         

    if err := UploadFile(bucket, region, filePath, key, acl); err != nil {
        fmt.Println("Error uploading file:", err)
    }
}

When I called the UploadFile function with the parameters, here’s proof that the file was uploaded to my S3 bucket.

You can always browse more details of files by clicking the file name on the AWS S3 console.

Downloading Files from an S3 Bucket

Now, you can attempt to retrieve the file with the key you’ve specified.

func DownloadFile(bucket, region, key, destPath string) error {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String(region),
    })
    if err != nil {
        return fmt.Errorf("error creating session: %w", err)
    }

    s3Client := s3.New(sess)

    output, err := s3Client.GetObject(&s3.GetObjectInput{
        Bucket: aws.String(bucket),
        Key:    aws.String(key),
    })
    if err != nil {
        return fmt.Errorf("error downloading file: %w", err)
    }
    defer output.Body.Close()

    destFile, err := os.Create(destPath)
    if err != nil {
        return fmt.Errorf("error creating destination file: %w", err)
    }
    defer destFile.Close()

    if _, err := io.Copy(destFile, output.Body); err != nil {
        return fmt.Errorf("error saving file: %w", err)
    }

    fmt.Println("File downloaded successfully:", key, "to", destPath)
    return nil
}

After creating the client instance, the GetObject function receives the file parameters in the &s3.GetObjectInput instance. You can now copy the file stream to your preferred destination with io.Copy function.

func main() {
    bucket := "cloudboxbucket"  
    region := "eu-north-1"      
    key := "Makefile"          
    destPath := "internal/database/DownloadedMakefile"

    if err := DownloadFile(bucket, region, key, destPath); err != nil {
        fmt.Println("Error downloading file:", err)
    }
}

The file should be downloaded to your specified destination path when executing a program with the DownloadFile function.

You’ll need to specify the key in the key in the destination path as well.

Deleting Files from S3 Buckets

To delete a file from an S3 bucket, you’ll use the AWS SDK’s DeleteObject function to remove the file. You must specify the S3 bucket name and the file key you want to delete.

func DeleteFile(bucket, region, key string) error {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String(region),
    })
    if err != nil {
        return fmt.Errorf("error creating session: %w", err)
    }

    s3Client := s3.New(sess)

    _, err = s3Client.DeleteObject(&s3.DeleteObjectInput{
        Bucket: aws.String(bucket),
        Key:    aws.String(key),
    })
    if err != nil {
        return fmt.Errorf("error deleting file: %w", err)
    }

    err = s3Client.WaitUntilObjectNotExists(&s3.HeadObjectInput{
        Bucket: aws.String(bucket),
        Key:    aws.String(key),
    })
    if err != nil {
        return fmt.Errorf("error waiting for file deletion: %w", err)
    }

    fmt.Println("File deleted successfully:", key)
    return nil
}

The DeleteFile function starts by creating a session, like in the upload and download examples. The s3Client.DeleteObject is called with a DeleteObjectInput struct, where you’ll specify the Bucket and Key parameters.

You can use the WaitUntilObjectNotExists function to confirm that the file doesn’t exist.

func main() {
    bucket := "cloudboxbucket"   
    region := "eu-north-1"       
    key := "Makefile"           

    if err := DeleteFile(bucket, region, key); err != nil {
        fmt.Println("Error deleting file:", err)
    }
}

When executing the DeleteFile function, you can see that the Makefile no longer exists in my S3 bucket.

Conclusion

You’ve learned how to set up an S3 bucket for an IAM user (it also works for root) and upload, download, and delete files from S3 buckets.

There’s so much more functionality you can do with AWS S3 Buckets. I hope this article has provided the foundation for you to go further, depending on your project’s use case.

How to Deploy Go Applications With Docker

Ukeje Chukwuemeriwo Goodness — Sun, 03 Nov 2024 20:24:58 +0000

Docker is a containerization platform that simplifies applications’ packaging, distribution, and deployment. You can harness the benefits of Go and Docker to enhance your applications’ efficiency, portability, and security.

This tutorial is invested in teaching you how you can build and deploy your Go applications with Docker. You’ll learn by building a RESTful API with the Gorilla Mux and GORM packages that you’ll containerize and deploy.

Step 1: Setting Up Your Development Environment

You’d need to have Go and Docker installed on your computer to build and containerize your Go apps with Docker.

Ensure that you have Go and Docker installed on your system. You can download Go from the official Go downloads website and Docker from Docker Hub. Visit the webpage if you haven’t, and follow the installation instructions for your specific operating system.

This article teaches how to deploy Go apps with Docker and teaches you more about installing and setting up Docker and a Postgres database, including containerizing your Go applications.

After installation, configure your Go development environment by setting environment variables and paths as needed. Ensure that you have a working Go workspace with the required directory structure.

Additionally, you can familiarize yourself with Docker's command-line interface (CLI) and basic Docker concepts.

Create a new directory for this project and run the go mod init command to initialize the directory as a Go project.

go mod init

After initializing the Go project, run this command to add the GORM and Gorilla Mux packages as dependencies to your project.

go get github.com/gorilla/mux

go get gorm.io/gorm
go get gorm.io/driver/postgres

You’ll use the Gorilla Mux package for routing. The GORM package provides an interface for you to use Go types for SQL database operations along with the driver package you installed (in this case, Postgres).

Step 2: Building the Go Application

In this tutorial, you’ll use a popular Go layered architecture style and use interfaces to interact with the various components of our app.

Here’s the directory structure of the application.

.
├── Dockerfile
├── cmd
│   └── server
│       └── main.go
└── internal
    ├── http
    │   ├── handlers.go
    │   └── users.go
    ├── models
    │   ├── database.go
    │   ├── migrations.go
    │   └── users.go
    └── users
        └── user.go

6 directories, 11 files

Dockerfile: The Dockerfile is the configuration file for building the Docker image. You’ll write the contents of this file based on the API.
cmd: The cmd directory usually contains the entry point of your application. The server sub-directory suggests that the API server is the main component of your project. The main.go file in the cmd/server contains the application's entry point.
internal: The internal directory helps organize the internal packages of your application. The internal packages shouldn’t be exported, making them an excellent place to hide implementation details.
- http: This subdirectory will contain HTTP-related code, including route handlers and possibly middleware for your API.
  - handlers.go: You’ll include your HTTP request handlers in this file.
  - users.go: You’ll specify HTTP handlers related to user management here.
- models: This directory will contain database-related code and data models.
- database.go: This file will contain code for initializing and connecting to the database.
- migrations.go: This file handles database schema migrations, ensuring that your database schema matches your application's requirements.
- users.go: This file will have data models or struct definitions related to users interacting with the database.
- users: This directory will contain user-specific logic.
  - user.go: This file includes functions and structures related to user management that interact with the database implementation. The HTTP implementation interacts with the functions here.

This project structure appears well-organized, clearly separating concerns between different components. This organization makes it easier to maintain and scale your Go API as it grows.

This isn’t a Go standard. However, many Go developers and open-source projects use this structure for your applications.

Step 2a: Writing the Database Implementation

You’ll set up database functionality for your application. You’ll have to define the models using structs, connect to the database, and set up migrations for your insertion operations on your database.

Here’s the list of imports you’ll need for the database implementation.

package models

import (
    // imports from the user implementation
    "BetterApp/internal/users"

    "context"
    "gorm.io/gorm"
    "fmt"
    "gorm.io/driver/postgres"
    "gorm.io/gorm/schema"
    "os"
)

The first task is defining a strut that matches your database schema for your app. GORM provides tags for specifying additional options and constraints on fields.

// internal/models/users.go

type User struct {
    gorm.Model
    Username string `gorm:"unique;not null"`
    Email    string `gorm:"unique;not null"`
    IsActive bool   `gorm:"not null"`
}

The User struct represents the model for working with user data in a database.

In your database.go file, declare a struct to encapsulate the database connection instance. You’ll use the struct to connect to your database from other parts of the database implementation package.

// internal/models/database.go

type Database struct {
    Client *gorm.DB
}

Next, create a database connection function that connects the database implementation to the database program to the database:

// internal/models/database.go

func NewDatabase() (*Database, error) {

    // Construct a connection string using environment variables for database configuration.
    configurations := fmt.Sprintf("host=%v port=%v user=%v password=%v dbname=%v sslmode=%v",
        os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_USERNAME"),
        os.Getenv("DB_PASSWORD"), os.Getenv("DB_NAME"), os.Getenv("SSL_MODE"))

    // Open a connection to the database using GORM and PostgreSQL driver.
    db, err := gorm.Open(postgres.New(postgres.Config{
        DSN:                  configurations,
        PreferSimpleProtocol: true,
    }), &gorm.Config{NamingStrategy: schema.NamingStrategy{
        SingularTable: true,
    }})
    if err != nil {
        return nil, err
    }

    // Enable connection pooling by configuring maximum idle and open connections.
    sqlDB, err := db.DB()
    if err != nil {
        return nil, err
    }
    sqlDB.SetMaxIdleConns(10)
    sqlDB.SetMaxOpenConns(100)

    // Return the Database instance with the established database connection.
    return &Database{
        Client: db,
    }, nil
}

The NewDatabase function creates a new Database instance and establishes a connection to the database. It returns a pointer to the Database instance, and an error, if any, occurs during the process.

After a successful database connection, you can set up migration functionality for your database implementation with the function as thus:

// internal/models/migrations.go

func (d *Database) MigrateDB() error {
    log.Println("Database Migration in Process...")

    // Use GORM AutoMigrate to migrate all the database schemas.
    err := d.Client.AutoMigrate(&User{})
    if err != nil {
        return err
    }

    log.Println("Database Migration Complete!")
    return nil
}

The MgrateDB function sets up automatic migrations for the User struct with the database client AutoMigrate function and returns an error if there’s any encountered during the process.

Step 2b: Defining Functions for the Database Implementation

In the users.go file where you defined the struct for your database schema, you can proceed to define the functions for the database implementation.

Here are the CreateUser, GetUserByID, UpdateUser, and DeleteUser functions responsible for CRUD operations on the database.

// internal/models/users.go

func (d *Database) CreateUser(ctx context.Context, user *users.User) error {
    newUser := &User{
        Username: user.Username,
        Email:    user.Email,
        IsActive: false,
    }

    if err := d.Client.WithContext(ctx).Create(newUser).Error; err != nil {
        return err
    }

    return nil
}

// GetUserByID returns the user with a specified id
func (d *Database) GetUserByID(ctx context.Context, id int64) (users.User, error) {
    user := users.User{}
    if err := d.Client.WithContext(ctx).Where("id = ?", id).First(&user).Error; err != nil {
        return users.User(User{}), err
    }
    return users.User(User{
        Username: user.Username,
        Email:    user.Email,
        IsActive: user.IsActive,
    }), nil
}

// UpdateUser updates an existing user in the database
func (d *Database) UpdateUser(ctx context.Context, updatedUser users.User, id uint) error {
    // Check if the user with the specified ID exists
    var existingUser User
    if err := d.Client.WithContext(ctx).Where("id = ?", id).First(&existingUser).Error; err != nil {
        return err
    }

    // Update the fields of the existing user with the new values
    existingUser.Username = updatedUser.Username
    existingUser.Email = updatedUser.Email
    existingUser.IsActive = updatedUser.IsActive

    // Save the updated user back to the database
    if err := d.Client.WithContext(ctx).Save(&existingUser).Error; err != nil {
        return err
    }

    return nil
}

// DeleteUser deletes a user from the database by their ID

func (d *Database) DeleteUser(ctx context.Context, id uint) error {
    // Check if the user with the specified ID exists
    var existingUser User
    if err := d.Client.WithContext(ctx).Where("id = ?", id).First(&existingUser).Error; err != nil {
        return err
    }

    // Delete the user from the database
    if err := d.Client.WithContext(ctx).Delete(&existingUser).Error; err != nil {
        return err
    }

    return nil
}

Your user implementation will call these functions to access the database functionality.

Step 2c: Writing the User Implementation

Your user implementation plays a significant role in relaying data from the database to the HTTP implementation.

You’ll define a struct that matches the struct in the database implementation and add JSON tags to the fields for usage; then, you’ll define functions that call the database functions with the data from the HTTP implementation.

Here are the imports you’ll need for your user implementation:

package users

import (
    "context"
    "gorm.io/gorm"
    "log"
)

Here’s the structure with JSON tags. The json:"-" in the gorm.Model field specifies that you want to exclude the field from the JSON operations.

// User -  a representation of the users of the wallet engine
type User struct {
    gorm.Model `json:"-"`
    Username   string `json:"username"`  
    Email      string `json:"email"`     
    IsActive   bool   `json:"is_active"` 
}

Next, you’ll declare an interface with methods for the user implementation functions, a service struct for the user implementation, and a function that initializes the service implementation.

type UserStore interface {
    CreateUser(context.Context, *User) error
    GetUserByID(context.Context, int64) (User, error)
    UpdateUser(context.Context, User, uint) error
    DeleteUser(context.Context, uint) error
}

// The UserService struct manages user-related operations outside the user implementation.
type UserService struct {
    Store UserStore // UserStore is an interface that abstracts the database operations.
}

// NewService creates a new service
func NewService(store UserStore) UserService {
    return UserService{
        Store: store,
    }
}

The interface and service will help manage user-related operations outside the user implementation.

Next, you can define methods of the UserService struct implementation that call the database implementation.


// CreateUser creates a new user by calling the CreateUser method on the associated UserStore.
func (u *UserService) CreateUser(ctx context.Context, user *User) error {
    if err := u.Store.CreateUser(ctx, user); err != nil {
        log.Printf("Error creating user: %v", err)
        return err
    }
    return nil
}

// GetUserByID retrieves a user by their ID using the GetUserByID method on the associated UserStore.
func (u *UserService) GetUserByID(ctx context.Context, id int64) (User, error) {
    user, err := u.Store.GetUserByID(ctx, id)
    if err != nil {
        log.Printf("Error fetching user with ID %v: %v", id, err)
        return user, err
    }
    return user, nil
}

// UpdateUser updates an existing user by calling the UpdateUser method on the associated UserStore.
func (u *UserService) UpdateUser(ctx context.Context, user User, id uint) error {
    if err := u.Store.UpdateUser(ctx, user, id); err != nil {
        log.Printf("Error updating user: %v", err)
        return err
    }

    return nil
}

// DeleteUser deletes a user by their ID using the DeleteUser method on the associated UserStore.
func (u *UserService) DeleteUser(ctx context.Context, id uint) error {
    if err := u.Store.DeleteUser(ctx, id); err != nil {
        log.Printf("Error deleting user: %v", err)
        return err
    }

    return nil
}

The CreateUser, GetUserByID, UpdateUser, and DeleteUser functions are responsible for calling the CRUD operations on the database implementation. The HTTP implementation will call these functions to access the database.

Step 2c: Writing the HTTP Implementation

The HTTP implementation is part of your application that receives and interacts with incoming requests.

Here’s the list of imports you’ll need across your HTTP implementation:

package http

import (
    "BetterApp/internal/users"
    "context"
    "github.com/gorilla/mux"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"
    "time"
    "encoding/json"
    "strconv"
)

First, declare a struct and include a Router instance, a HTTP instance, and an instance of the user service.

// internal/http/handler.go

type Handler struct {
    Router *mux.Router
    Users  users.UserService
    Server *http.Server
}

Then create a function that returns a pointer to the Handler struct, where you can configure the server and handlers.

// internal/http/handler.go

// NewHandler - returns a pointer to a Handler
func NewHandler(users users.UserService) *Handler {
    // Print a log message to indicate that we are setting up our handler
    log.Println("setting up our handler")

    // Create a new Handler instance and initialize it
    h := &Handler{
        Users: users, // Assign the provided users service to the handler's Users field
    }

    // Create a new HTTP router using the mux package
    h.Router = mux.NewRouter()

    // Map the routes for this handler (assuming this method exists and sets up routes)
    h.mapRoutes()

    // Create an HTTP server instance with specified settings
    h.Server = &http.Server{
        Addr:         "0.0.0.0:8080", // Set the server address to listen on
        WriteTimeout: time.Second * 15, // Set the maximum time to write the response
        ReadTimeout:  time.Second * 15, // Set the maximum time to read the request
        IdleTimeout:  time.Second * 60, // Set the maximum idle time
        Handler:      h.Router, // Set the router as the handler for incoming requests
    }

    // Return the initialized handler
    return h
}

The NewHandler function sets up and configures an HTTP request handler, making it ready to handle incoming HTTP requests for a specific service while also defining server settings and routes.

The mapRoutes function you called in the NewHandler function sets up routes by mapping them to their respective handler functions.

// internal/http/handler.go

// mapRoutes - sets up all the routes for our application
func (h *Handler) mapRoutes() {
    h.Router.HandleFunc("/api/v1/user/create", h.CreateUser).Methods("POST")
    h.Router.HandleFunc("/api/v1/users/{id}", h.GetUserByID).Methods("GET")
    h.Router.HandleFunc("/api/v1/users/{id}", h.UpdateUser).Methods("PUT")
    h.Router.HandleFunc("/api/v1/users/{id}", h.DeleteUser).Methods("DELETE")
}

Next, define the handler functions and their functionalities. Here are the CreateUser, GetUserByID, UpdateUser and DeleteUser functions that are responsible for intercepting HTTP requests and responding based on the operation.

// internal/http/users.go

// CreateUser decodes a User object from the HTTP request body and then tries to create a new user in the database using the CreateUser method of the UserService interface.
// If the user is successfully created, it encodes and sends the created user as a response.
func (h *Handler) CreateUser(writer http.ResponseWriter, request *http.Request) {
    // Decode the incoming JSON request body into a User struct
    var u users.User
    if err := json.NewDecoder(request.Body).Decode(&u); err != nil {
        // If decoding fails, respond with a Bad Request status and log the error
        http.Error(writer, "Failed to decode request body", http.StatusBadRequest)
        log.Println("Failed to decode request body:", err)
        return
    }

    // Try to create the user using the CreateUser method of UserService
    err := h.Users.CreateUser(request.Context(), &u)
    if err != nil {
        // If user creation fails, respond with an Internal Server Error status and log the error
        http.Error(writer, "Failed to create user", http.StatusInternalServerError)
        log.Println("Failed to create user:", err)
        return
    }

    // Set the response content type to JSON, set the status to Created, and encode and send the created user as the response
    writer.Header().Set("Content-Type", "application/json")
    writer.WriteHeader(http.StatusCreated)

    if err := json.NewEncoder(writer).Encode(u); err != nil {
        // If encoding the response fails, log the error (Panicln is used for critical errors)
        log.Panicln("Failed to encode response:", err)
    }
}

// GetUserByID extracts the id from the URL parameters and then fetches the user with that id from the database using the GetUserByID method of the UserService interface.
// If the user is found, it encodes and sends the user as a response.
func (h *Handler) GetUserByID(writer http.ResponseWriter, request *http.Request) {
    // Extract the 'id' parameter from the URL and convert it to an integer
    vars := mux.Vars(request)
    id, err := strconv.ParseInt(vars["id"], 10, 64)
    if err != nil {
        // If parsing the 'id' fails, respond with a Bad Request status and the error message
        http.Error(writer, err.Error(), http.StatusBadRequest)
        return
    }

    // Fetch the user with the given 'id' using the GetUserByID method of UserService
    u, err := h.Users.GetUserByID(request.Context(), id)
    if err != nil {
        // If fetching the user fails, log the error
        log.Println(err)
        return
    }

    // Encode and send the fetched user as the response
    if err := json.NewEncoder(writer).Encode(u); err != nil {
        // If encoding the response fails, log the error (Panicln is used for critical errors)
        log.Panicln(err)
    }
}

// UpdateUser updates a user by ID.
func (h *Handler) UpdateUser(writer http.ResponseWriter, request *http.Request) {
    // Extract the 'id' parameter from the URL and convert it to an integer
    vars := mux.Vars(request)
    id, err := strconv.ParseInt(vars["id"], 10, 64)
    if err != nil {
        // If parsing the 'id' fails, respond with a Bad Request status and the error message
        http.Error(writer, err.Error(), http.StatusBadRequest)
        return
    }

    // Decode the request body to get the updated user information
    var u users.User
    if err := json.NewDecoder(request.Body).Decode(&u); err != nil {
        // If decoding the request body fails, respond with a Bad Request status
        http.Error(writer, "Invalid request body", http.StatusBadRequest)
        return
    }

    // Update the user using the UpdateUser method of UserService
    err = h.Users.UpdateUser(request.Context(), u, uint(id))
    if err != nil {
        // If updating the user fails, respond with an Internal Server Error status and log the error
        http.Error(writer, "Failed to update user", http.StatusInternalServerError)
        log.Println(err)
        return
    }

    // Encode and send the updated user as the response
    if err := json.NewEncoder(writer).Encode(u); err != nil {
        // If encoding the response fails, respond with an Internal Server Error status and log the error (Panicln is used for critical errors)
        http.Error(writer, "Failed to encode response", http.StatusInternalServerError)
        log.Panicln(err)
    }
}

// DeleteUser deletes a user by ID.
func (h *Handler) DeleteUser(writer http.ResponseWriter, request *http.Request) {
    // Extract the 'id' parameter from the URL and convert it to an integer
    vars := mux.Vars(request)
    id, err := strconv.ParseInt(vars["id"], 10, 64)
    if err != nil {
        // If parsing the 'id' fails, respond with a Bad Request status and the error message
        http.Error(writer, err.Error(), http.StatusBadRequest)
        return
    }

    // Delete the user using the DeleteUser method of UserService
    err = h.Users.DeleteUser(request.Context(), uint(id))
    if err != nil {
        // If deleting the user fails, respond with an Internal Server Error status and log the error
        http.Error(writer, "Failed to delete user", http.StatusInternalServerError)
        log.Println(err)
        return
    }

    // Send a success response with a No Content status (204)
    writer.WriteHeader(http.StatusNoContent)
}

Now, you can write the functionality for starting the server.

// internal/http/handler.go

// Serve - gracefully serves our newly set up handler function
func (h *Handler) Serve() error {
        if err := h.Server.ListenAndServe(); err != nil {
            log.Println(err)
        }
}

The Serve function starts the server on the specified port and returns an error if there’s any during the process.

Step 2d: Coupling the Implementations and Running the Application

Import the implementations in your main.go file to couple the implementations and run your app.


package main

import (
    "BetterApp/internal/http"
    "BetterApp/internal/models"
    "BetterApp/internal/users"
    "fmt"
    "log"
)

You can declare a Run function that instantiates the startup of your app in the main.go file and then call the function in the main function.

//cmd/server/main.go

// Run - is going to be responsible for / the instantiation and startup of our / go application
func Run() error {
    fmt.Println("starting up the application...")

    // create a database instance variable
    store, err := models.NewDatabase()
    if err != nil {
        log.Println("Database Connection Failure")
        return err
    }

    // initialize the migrations functionality on the new database
    if err := store.MigrateDB(); err != nil {
        log.Println("failed to setup store migrations")
        return err
    }

    // set the database instance as the store for the user service implementation
    userService := users.NewService(store)

    // initialize a new handler with the user service 
    handler := http.NewHandler(userService)

    // call the serve function to start the server
    if err := handler.Serve(); err != nil {
        log.Println("failed to gracefully serve our application")
        return err
    }

    return nil

}

The Run function creates a database instance, initializes migrations functionality, initializes the HTTP and User implementations and starts the server.

You can call the Run function in the main function to launch your application.

//cmd/server/main.go

func main() {
    fmt.Println("Go REST API Tutorial")
    if err := Run(); err != nil {
        log.Println(err)
    }

}

The application should run fine before you consider containerizing it with Docker.

Step3: Writing the Dockerfile

Now that you’ve successfully built and ran the program, you can proceed to containerize it with Docker.

Your Dockerfile will have two stages, the build and final stage. This approach reduces image size, minimizes security risks by reducing the attack surface, ensures efficient runtime performance, and facilitates reproducibility across different development and deployment stages.

You’ll also use Alpine Linux as the base image of your Docker images since they’re more efficient and secure with minimalist design results in smaller image sizes, faster builds, and reduced attack surfaces.

Step 3a: The Build Stage

Using the build and final stages in a Dockerfile allows for the efficient creation of Docker images. The build stage starts with a base image containing build tools and dependencies, compiles application artifacts, and generates a potentially large intermediate image.

Here’s the contents of the Dockerfile for the Build Stage:

# Build stage
FROM golang:1.20-alpine AS build

# Set the working directory inside the container
WORKDIR /app

# Copy the Go application source code into the container
COPY . .

# Build the Go application
RUN go build -o server ./cmd/server

FROM golang:1.20-alpine AS build: This line specifies the base image for the build stage. It starts with the official Golang Docker image tagged with version 1.20 and is based on Alpine Linux. The AS build part gives this stage a name, "build" that you can reference later.
WORKDIR /app: This line sets the working directory inside the container to /app. Docker will execute subsequent commands in this directory.
COPY . .: This command copies the contents of the current directory (presumably your Go application source code and other necessary files) into the /app directory inside the container.
RUN go build -o server ./cmd/server: This is the command that builds the Go application. It uses the go build command to compile the Go code in the current directory and output the binary as server. The ./cmd/server argument is the location of the application code relative to the /app directory.

Step 3b: The Final Stage

The final stage employs a smaller base image, copies only necessary runtime components, and results in a compact image optimized for production.

Here are the contents of your Dockerfile for the final stage:

# Final stage
FROM alpine:latest

# Set the working directory inside the final container
WORKDIR /app

# Copy the binary built in the previous stage
COPY --from=build /app/server .

# Expose the port your application will listen on (adjust as needed)
EXPOSE 8080

# Run your Go application
CMD ["./server"]

FROM alpine:latest: In the final stage, you can start with an Alpine Linux base image, The latest tag specifies the latest available version of Alpine Linux.
WORKDIR /app: This line sets the working directory inside the container to /app. Docker will execute subsequent commands in this directory.
COPY --from=build /app/server .: This command copies the binary file named server from the previous "build stage" into the /app directory inside the final container. This binary is the compiled Go application that you built in the build stage.
EXPOSE 8080: Here, you specify that your application will listen on port 8080. This is a declaration and does not actually open the port; it's a way to document which port your application expects to use.
CMD ["./server"]: This command will be executed when you run a container based on the image. It specifies running the server binary, which is your Go application. This command starts your application inside the container.

Step 4: Building and Running the Docker Image

After Writing the Dockerfile, you can proceed to build and run the file.
Run this command to build the Docker image from the file with the build command.

docker build -t betterapp .

The -t flag specifies the tag for the Docker image as betterapp and the following dot (.) specifies that you want to build the Dockerfile in the current directory.

You can run the image with the run command and specify a port mapping from the container to your host machine with the -p flag.

docker run -p 8080:8080 \
  -e DB_HOST=ep-silent-dew-52667493-pooler.us-east-2.aws.neon.tech \
  -e DB_PORT=5432 \
  -e DB_USERNAME=Goodnessuc \
  -e DB_PASSWORD=foOEky9F2reG \
  -e DB_NAME=neondb \
  -e SSL_MODE=require \
  betterapp

The subsequent -e flags are for specifying environment variables since for your application.

Step 5: Deploying Go Applications with Docker

Docker Compose is a container orchestration tool that simplifies working with multiple Docker containers. You can use Docker compose to orchestrate your Go apps and their components.

You’ll use a YAML file to specify the instruction and Docker compose will setup your applications to save you time and complexity.

First, create a Docker Compose YAML file with the command below and open the file in your editor:

touch docker-compose.yml

After creating the Dockerfile, you can start writing the commands and directives for deploying your app:

version: "3.8"

services:
  db:
    image: postgres:12.2-alpine
    container_name: "betterapp"
    ports:
      - "5433:5432"
    environment:
      - POSTGRES_DB=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    networks:
      - fullstack
    volumes:
      - database_postgres:/var/lib/postgresql/data

  api:
    build: .
    container_name: "betterapp-rest-api"
    environment:
      DB_USERNAME: "postgres"
      DB_PASSWORD: "postgres"
      DB_DB: "postgres"
      DB_HOST: "db"
      DB_TABLE: "postgres"
      DB_PORT: "5432"
      SSL_MODE: "disable"
    ports:
      - "8080:8080"
    depends_on:
      - db
    networks:
      - fullstack

volumes:
  database_postgres:

networks:
  fullstack:
    driver: bridge

The YAML file defines two services: my-postgres which is the database container instance and the web service, which is your Go application before configuring their environment variables, ports, and dependencies.

Now, you can proceed to build the images with the docker-compose build command.

docker-compose build

Your output should be similar to this:

Finally, you can run your containers with the docker-compose up command.

docker-compose up -d

The -d flag runs the containers in detached mode which makes it agnostic of the terminal session.

Here’s the result of running the command:

You can close your terminal, and the container should continue running.

You can run the CURL requests to test your API once the containers are up:

# Create a new user
# POST /api/v1/user/create
curl -X POST \
     -H "Content-Type: application/json" \
     -d '{"username": "alice_smith", "email": "alice.smith@mydomain.com", "is_active": true}' \
     http://localhost:8080/api/v1/user/create

# Get user by ID
# Assuming the user ID of Alice Smith is 2
# GET /api/v1/users/{id}
curl -X GET http://localhost:8080/api/v1/users/1

# Update user by ID
# Let's update Alice Smith's username and email address
# PUT /api/v1/users/{id}
curl -X PUT \
     -H "Content-Type: application/json" \
     -d '{"username": "alice_wonder", "email": "alice.wonder@wonderland.com", "is_active": true}' \
     http://localhost:8080/api/v1/users/1

# Delete user by ID
# DELETE /api/v1/users/{id}
curl -X DELETE http://localhost:8080/api/v1/users/1

Congratulations, you’ve successfully deployed and run a working Go app with Docker and Docker Compose.

Conclusion

You’ve learned how to build and simplify your Go app’s deployment with Docker and Docker Compose. As you continue on your development journey, the skills and understanding you've gained here will prove to be essential assets in ensuring smooth deployments and operational excellence.

Consider exploring advanced Docker features like optimizing Dockerfile builds or implementing Docker Swarm for larger applications.

# Now, You Understand Blockchain Interoperability

Ukeje Chukwuemeriwo Goodness — Thu, 31 Oct 2024 23:47:20 +0000

Interoperability has been a big deal for some time, just as it was in the early days of networking and many other technologies.

Still, unlike traditional technologies, permissionlessness makes it hard to standardize, and now, there’s a colony of approaches, each with its pros and cons for interoperability on different layers.

This article is your last stop. You’ll finally understand blockchain interoperability, learn its fundamentals, and how different stacks work and compare.

State of The Blockchain Interoperability Problem

Blockchains are trust-optimized systems. From a high-level overview, the problem is simple: Different networks have different architectures and run on different tech stacks, consensus mechanisms and protocols that cannot communicate directly. There’s also:

Different Consensus Mechanisms: Blockchains validate their transactions differently to achieve consensus, which can be challenging, especially with trusted data from third-party chains.
Different Data Structures: Bitcoin's Unspent Transaction Output (UTXO) model contrasts Ethereum's account-based model and Sui’s object model, so supporting the same data on other chains would be generic.
Smart Contract Compatibility: The different virtual machines execute smart contracts based on the primitives they’re built on. Executing smart contracts is a big part of blockchain interoperability, and since there’s no standard environment, there are limits.
Token Standards: ERC-20 on Ethereum and CW-20 on Cosmos are far from similar, and wrapping and pegging are risky.

...and many other issues.

Many projects are building solutions to these issues, from bridges to sidechains, relay chains with shared security, atomic swaps, interoperability protocols, and networks of networks.

You can classify the systems of an interoperability solution into four layers/components.

The Fundamental Technical Layers: Messaging, Transport, and Consensus Layers.
Security and Validation Components: Relayers, Validators, Oracles, and Verifiers.
Cross-Chain Operations: Wrapped Tokens, Multisig Wallets, and Cross-Chain Liquidity.
Innovative Mechanisms: Zero-Knowledge Proofs, State Synchronization, and Event Listeners.

The different interoperability solutions have different components. Bridges, Interoperability Protocols, and cross-chain messaging protocols are the most prominent interoperability solutions. Let’s overview how each of these works to achieve cross-chain operations, down to their implementation aspects like light clients, relayers, validator networks, liquidity pools, and messaging frameworks.

Bridges

Bridges are interoperability solutions that connect L1<>L2 and L2<>L2 and L1<>L1 communication, usually running on similar virtual machines, e.g. EVM chains.

There are two types of bridges: trusted bridges and trust-minimized bridges.

Trusted Bridges rely on centralized parties to verify and execute the operation, e.g. (Axelar and IBC).
Trust-minimized bridges, such as the Optimism to ETH official bridge, use on-chain mechanisms like fraud and validity proofs to verify transactions.

Most bridges use smart contracts on the Lock-and-Release or Burn-and-Mint Mechanism, where the bridge locks or burns the asset on the originating chain and mints or releases the same asset on the corresponding chain. For example, Circle CCTP uses the burn-and-mint mechanism, while Orbiter Finance uses lock-and-release.

Relayers are the intermediaries between the chains. They confirm the asset's status on the originating chain and update the bridge’s smart contract on the destination chain. Relayers usually run nodes on both chains to verify state changes or initiate transactions.

Trust-minimized bridges like Axelar also use a validator set for verifiability and more security. The number and security of the validators impact the bridge’s security.

Interoperability Protocols

Interoperability protocols provide a shared protocol layer for blockchains to connect to multiple blockchains and communicate natively, regardless of their primitive features.

Most interoperability protocols are networks of networks, e.g. Polkadot and Cosmos. Cosmos uses IBC (Inter-Blockchain Communication), while Polkadot relies on a relay chain and parachains.

Interoperability protocols use light clients that run on every chain in the network to verify the state of another chain. These light clients hold a simplified version of the chain’s state and verify block headers to confirm transactions.

On Polkadot, a relay chain connects multiple parachains (independent chains) and handles consensus, security and messaging. Collators in parachains gather transactions and produce blocks, while validators on the relay chain finalize the blocks.

Cosmos’s IBC Protocol uses a handshake mechanism to establish secure communication between blockchains. Each chain must have IBC modules set with light clients

Cross-Chain Messaging Protocols

Cross-chain messaging protocols are great for secure messaging across blockchains with different virtual machines. They’d usually allow smart contract calls, data exchange and complex cross-chain apps.

Popular solutions in this category are Axelar, LayerZero, Wormhole, Union, and Hyperlane. Axelar provides secure cross-chain messaging and transaction interoperability through a decentralized validator network, and Wormhole uses a guardian network to observe, validate, and relay messages.

These cross-chain messaging protocols transmit data between chains through standard messaging formats that contain instructions (e.g. to execute a smart contract on another chain) or any data.

Some protocols use external validators or Oracle-based verification systems to confirm the state of the source and destination chains. They also use protocol-specific relayers and messaging agents to send messages and submit validity proofs to contracts on the destination chain.

Also, messaging contracts on each chain process incoming messages to determine whether they’re valid before executing them after the validators and guardians have signed them.

Comparing Interoperability Solutions

Interoperability solutions vary in architecture, security models, core components implementation and many other aspects.

Here’s a table that overviews the three categories with more insight into their aspects.

Aspect	Bridges	Interoperability Protocols	Cross-Chain Messaging
Examples	Circle CCTP, Orbiter Finance, Optimism Bridge	Cosmos (IBC), Polkadot (Relay Chain)	Wormhole, LayerZero, Axelar, Hyperlane
Architecture	Point-to-point connections between chains; Lock-and-Release or Burn-and-Mint pattern	Hub-and-spoke or network-of-networks with shared security	Universal messaging layer with standardized formats
Security Model	Trusted (centralized) or Trust-minimized (on-chain verification)	Shared security model with validator networks	Hybrid model with guardian networks/oracle systems
Core Components	Smart contracts, Relayers, Token wrapping mechanisms	Light, clients, Relay chain/hub, Protocol modules	Messaging contracts, Validator networks, Standard interfaces
Scalability	n² relationship (linear growth with new chains)	n relationship (efficient with new chains)	Highly scalable, minimal additional infrastructure
Implementation	Medium effort, high maintenance per bridge	High initial effort, lower subsequent integration	Medium effort, low maintenance
Best Use Cases	Direct asset transfers, simple cross-chain transactions	Ecosystem-wide connectivity, complex applications	Universal messaging, cross-ecosystem applications
Limitations	Limited scalability, higher security risks	Protocol lock-in, high initial setup cost	Message complexity, reliance on external networks

The wheel is the best of its kind, so nobody is trying to reinvent it; in the same vein, every interoperability solution has its strong foot and Achilles heel.

Bridges are great! but they cannot transfer messages using interoperability protocols and cross-chain messaging apps.

Interoperability and cross-chain messaging protocols have bridges, and they can send messages and perform more complex operations, such as executing smart contracts and managing governance across chains.

Conclusion

Now, You Understand Blockchain Interoperability. If you’re building a project, you should overview these features and choose the best for your app, or use a bridge aggregator like LIFI if you’re building DeFi apps.

Interoperability is just getting started. We’re seeing features like restaking and tokenization take-off powered by interoperability between chains. I think with great interoperability, chains can focus on their strengths, and everyone wins.

OAuth Tutorial with Go and the Spotify API

Ukeje Chukwuemeriwo Goodness — Wed, 30 Oct 2024 04:04:47 +0000

When OAuth was released, it solved a major issue with application security, privacy, and user experience by allowing builders access to user data from external apps. OAuth2 has complexities that you’ll need to understand at least at a high level to integrate external services requiring OAuth2 in your apps.

This article will help you understand OAuth2 so that you can build and integrate external services that allow users to sign up and sign in with their external profiles, such as GitHub, Spotify, Google, etc.

Understanding the OAuth Flow

OAuth has a set of key components, parameters, and a different flow you’ll need to understand before building; let’s review them.

OAuth Concepts and Parameters

The Authorization Server handles user authentication and grants permissions to the client, in this case, Spotify.
The Resource Server is the server that hosts the data you want to access from the provider, in this case, Spotify’s API endpoints.
The Client is the app you’re building to make requests to the authorization and resource servers.
The Resource Owner owns the data you need to access from the OAuth provider.
The OAuth provider issues a Client ID and Client Secret to identify your app. The Client ID identifies your app, while the Client Secret is for authentication.
The Authorization Code is a short-lived code that the authorization server issues the client.
The authorization server also issues an Access Token for your client to access the resource.
The Refresh Token is long-lived, so your client can get new tokens from the authorization server when they expire without the user needing to authenticate again.
You’ll provide a Redirect URI to the OAuth provider so the authorization server can redirect the resource owner after they’ve authenticated and you’re authorized or they reject the request.
You’ll specify the Scopes in your request to define the resources you want to access. The OAuth provider would notify the resource owner of the scopes before they authenticate.
You must include a State Parameter in your request to maintain the state between the authorization request and callback. This step helps prevent Cross-Site Request Forgery (CSRF) attacks.

The OAuth2 Authorization Flow

When the user tries to give you access to an external resource, you need to direct them to the authorization server with a request that includes the parameters to indicate the data you need from the resource server.
The user reserves the right to grant or deny access. When they do, the authorization server will redirect them to the redirect URI with or without the authorization code.
If the user grants the client access, the client sends the authorization code and credentials to the authorization server in exchange for an access token.
Now, you can use the Access Token to access the user’s resource on the resource server.

Setup and Prerequisites

This article requires basic knowledge of Go, but it can also be read in other languages.

Since we are using the Spotify API for demonstration, you must set up an account to retrieve a Client ID and Client Secret for authorization. Remember to set the callback address to which Spotify will redirect users.

Now, initialize a new project and install the oauth2 package you’ll use for authorization.

go mod init
go get golang.org/x/oauth2

Finally, import these packages into your main.go file. Aside from the oauth2 package, every package on the list is part of the Go standard library.

import (
    "context"
    "encoding/json"
    "fmt"
    "golang.org/x/oauth2"
    "golang.org/x/oauth2/spotify"
    "log"
    "math/rand"
    "net/http"
    "time"
)

Before you start authorization with Oauth, you’ll need to understand how it all works; let’s get to it.

Implementing OAuth in Go

Now that you’re all set up, define a struct for the OAuth service. When creating a new service, you’ll initialize OAuth configurations with *oauth2.Config.

type OAuthService struct {
    config *oauth2.Config
    state  string
}

The state variable will be the state parameter; you’ll have something secure that you can always verify.

You can use the rand.Seed function to seed a random 64-it integer randomized by the current time.

func generateRandomState() string {
    rand.Seed(time.Now().UnixNano())
    return fmt.Sprintf("%d", rand.Intn(100000)) 
}

Calling rand.Intn(100000) sets the upper limit of the generated random number.

Now, you can define a function that will initialize a new OAuth service using the necessary parameters.

func NewOAuthService() *OAuthService {
    return &OAuthService{
        config: &oauth2.Config{
            ClientID:     "your client id",
            ClientSecret: "<your client secret",
            Endpoint:     spotify.Endpoint,
            RedirectURL:  "http://localhost:8080/callback/spotify",
            Scopes:       []string{"user-read-email", "user-read-recently-played"},
        },
        state: generateRandomState(), // Use a dynamic state in production
    }
}

It’s important to enter the right RedirectURL you provided to the OAuth provider and define only the necessary scopes.

You’ll need a login function that directs the users to the OAuth provider to initiate the OAuth process.

func (o *OAuthService) HandleSpotifyLogin(w http.ResponseWriter, r *http.Request) {
    url := o.config.AuthCodeURL(o.state)
    http.Redirect(w, r, url, http.StatusTemporaryRedirect)
}

Here, AuthCodeURL generates the URL and appends the state parameter before temporarily redirecting the user.

Now, you’ll need a function that handles callbacks from the OAuth provider. After extracting and validating state parameters, the function should redirect users to your app.

func (o *OAuthService) HandleCallback(w http.ResponseWriter, r *http.Request) {
    state := r.URL.Query().Get("state")
    code := r.URL.Query().Get("code")
    if state != o.state {
        http.Error(w, "Invalid state", http.StatusBadRequest)
        return
    }

    token, err := o.config.Exchange(context.Background(), code)
    if err != nil {
        http.Error(w, "Failed to exchange token", http.StatusInternalServerError)
        log.Println("Token exchange error:", err)
        return
    }

    client := o.config.Client(context.Background(), token)
    userInfo, err := getSpotifyUserInfo(client)
    if err != nil {
        http.Error(w, "Failed to get user info", http.StatusInternalServerError)
        log.Println("User info error:", err)
        return
    }
    fmt.Fprintf(w, "Logged in successfully! User: %s\n", userInfo.Name)

    playlistID := "37i9dQZF1EVHGWrwldPRtj" // Replace with the actual playlist ID you want to fetch
    tracks, err := getPlaylistTracks(client, playlistID)
    if err != nil {
        http.Error(w, "Failed to get playlist tracks", http.StatusInternalServerError)
        log.Println("Playlist tracks error:", err)
        return
    }

    for _, trackItem := range tracks {
        fmt.Fprintf(w, "Track: %s by %s (Album: %s)\n", trackItem.Track.Name, trackItem.Track.Artists[0].Name, trackItem.Track.Album.Name)
    }
}

First, the HandleCallback extracts the code and state parameters and validates the state. The Exchange method requests for an access token with the code parameter.

Finally, the Client function returns an HTTP client with the access token. The token will auto-refresh as necessary. You can now request the OAuth provider API from the client instance, as I have done with the getSpotifyUserInfo function call.

func getSpotifyUserInfo(client *http.Client) (string, error) {
    resp, err := client.Get("https://api.spotify.com/v1/me")
    if err != nil {
        return "", fmt.Errorf("failed to get user info: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return "", fmt.Errorf("error response from Spotify: %s", resp.Status)
    }

    var user struct {
        Name string `json:"display_name"`
    }
    if err := json.NewDecoder(resp.Body).Decode(&user); err != nil {
        return "", fmt.Errorf("failed to decode user info: %w", err)
    }

    return user.Name, nil
}

The getSpotifyUserInfo function accepts an HTTP client instance and makes a GET request to a Spotify API endpoint that requires OAuth for access. The client from the HandleCallback function is the sauce that makes the request valid.

Finally, you need to call the OAuth service, register the routes and start a server.

func main() {
    oauthService := NewOAuthService()

    http.HandleFunc("/login/spotify", oauthService.HandleSpotifyLogin)
    http.HandleFunc("/callback/spotify", oauthService.HandleCallback)

    fmt.Println("Server is running on http://localhost:8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

When running this program and following through with the authentication process, you should get a message with your username displayed on the browser.

Best Practices to Consider

When implementing an OAuth provider, follow this safety and great UX practices.

Use and enforce HTTPS to protect tokens from attackers.
Use secure storage solutions if you’re storing tokens.
Implement token expiration and automatic refreshes for good UX.
Minimize scopes and permissions to what’s necessary.
Do not store tokens in local storage for web apps.
Rotate, and Invalidate Tokens When compromised or the user has logged out.
Implement CSRF Protection with state parameters.

The data you get from OAuth providers is sensitive, so they specifically require OAuth on those endpoints. Security first!

Conclusion

Now that you know the basics of OAuth and how to implement OAuth providers and best practices, the next step is to build the other parts of your apps.

Use this foundation to explore other features like refresh tokens for sessions with longer lives, and always consider the best practices. Happy building.

How to Use the Sui TypeScript SDK

Ukeje Chukwuemeriwo Goodness — Sat, 26 Oct 2024 06:01:33 +0000

Sui is the chain of the moment, and although Move is the holy grail for writing smart contracts on Sui, the TypeScript support is significant. You can interact with and use Sui and most of the DeFi apps on the ecosystem with TypeScript.

In this tutorial, I’ll teach you how to interact with the Sui network via TypeScript. You’ll learn how to read the state of the blockchain, write transactions to the chain from your TypeScript programs.

Getting Started With Sui and TypeScript

The only prerequisite is that you’ll need basic JS/TS knowledge to run this tutorial smoothly. I’ll walk you through everything else.

First, create a new TypeScript project in your terminal and initialize a new Node.js project.

mkdir SuiTS
cd SuiTS
npm init -y

Install TypeScript as a development dependency if you don’t have it already.

npm install typescript --save-dev
npm install ts-node //runs TS without the need for transpiling

Now, you can initialize a new TypeScript project. This command will create a tsconfig.json file with default options that you can customize for your project.

npx tsc --init

Open the tsconfig.json and paste these configurations.

{
  "compilerOptions": {
    "target": "ES2020",
    "module": "CommonJS",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "types": ["node"],
    "resolveJsonModule": true
  },
  "exclude": ["node_modules"],

  "scripts": {
    "build": "tsc",
    "start": "node dist/index.js"
  }
}

Create a src directory where you’ll add your TypeScript files.

mkdir src
touch src/index.ts

Finally, install the Sui TypeScript SDK with this command.

npm i @mysten/sui.js

You’re all set. You can start writing TypeScript programs that interact with the Sui blockchain.

Connecting to the Sui Blockchain

You must connect to a Sui blockchain to interact with the chain.

First, import getFullnodeUrl and SuiClient from the SDK client module.

import { getFullnodeUrl, SuiClient } from '@mysten/sui/client';

Now, depending on your desired connection, you can use getFullnodeUrl to retrieve the full node URL of the Sui testnet, mainnet, localnet, or devnet; then, use the SuiClient to connect to the client instance.

import { getFullnodeUrl, SuiClient } from '@mysten/sui/client';

const rpcUrl = getFullnodeUrl('mainnet');

const client = new SuiClient({ url: rpcUrl });

To test your connection, You can use the getLatestSuiSystemState to retrieve the latest state of the network.

// index.ts
import { getFullnodeUrl, SuiClient } from '@mysten/sui/client';

const rpcUrl = getFullnodeUrl("mainnet");

const client = new SuiClient({ url: rpcUrl });

async function getNetworkStatus() {
    const currentEpoch = await client.getLatestSuiSystemState();
    console.log(currentEpoch)
}

getNetworkStatus();

Now, transpile the TypeScript code to JavaScript and run it with this command:

ts-node index.ts

You should expect an output similar to this when you execute the command.

Creating a Sui Wallet

Creating a wallet is another popular operation that might be handy if you build on the Sui Network.

Here’s how to generate Sui wallet keypairs and retrieve the private and public keys from the Keypair.

import { Ed25519Keypair } from '@mysten/sui/keypairs/ed25519';
import { getFullnodeUrl, SuiClient } from '@mysten/sui/client';

const rpcUrl = getFullnodeUrl("mainnet");

const client = new SuiClient({ url: rpcUrl });

// random Keypair
const keypair = new Ed25519Keypair();

const publicKey = keypair.getPublicKey();
const privatekey = keypair.getSecretKey();

console.log(privatekey.toString());
console.log(publicKey.toSuiAddress());

The Ed25519Keypair function creates a new key pair. The getPublicKey and getPrivateKey methods give you access to the public and private keys, respectively.

Here’s the string output of the private and public keys I generated with the program:

suiprivkey1qq9r6rkysny207t5vr7m5025swh7w0wzra9p0553paprhn8zshqsx2rz64r
New Sui Address: 0xbd46d7582ced464ef369114252704b10317436ef70f196a33fcf2c724991fcba

I’m funding this wallet with 0.25 Sui for the next set of operations. Feel free to verify and scan the wallet. Do not send any funds; it’s just a dummy wallet.

Reading Sui Wallet Balances

You can use the getCoins function on your client instance to retrieve details on the coins in a wallet address.

import { getFullnodeUrl, SuiClient } from '@mysten/sui/client';

// use getFullnodeUrl to define the Devnet RPC location
const rpcUrl = getFullnodeUrl('mainnet');

// create a client connected to devnet
const client = new SuiClient({ url: rpcUrl });

async function getOwnedCoins() {
    const coins = await client.getCoins({
        owner: '0xbd46d7582ced464ef369114252704b10317436ef70f196a33fcf2c724991fcba',
    });
    console.log(coins);
}

getOwnedCoins();

The function returns details on the Sui coin alone and the details. The output is in MIST, the Sui gas token. 1 SUI equals 1 billion MIST.

The getAllCoins function can be used in the same way to get a list of all the coins in a wallet.

async function getAllCoins() {
    // Get the list of owned coins (tokens) for the given owner address
    const ownedCoins = await client.getAllCoins({ owner: "0xbd46d7582ced464ef369114252704b10317436ef70f196a33fcf2c724991fcba" });

    // Access the coin data
    const coins = ownedCoins.data;

    // Iterate through the coins and print their details
    for (const coin of coins) {
        console.log(`Coin Type: ${coin.coinType}`);
        console.log(`Coin Object ID: ${coin.coinObjectId}`);
        console.log(`Balance: ${coin.balance}`);
        console.log('--------------------');
    }

    // If there is more data, handle pagination
    if (ownedCoins.hasNextPage) {
        console.log('More data available. Fetching next page...');
        // You can handle the next page using ownedCoins.nextCursor if needed
    }
}

getAllCoins();

For this example, I traded some Sui for $FUD on Hop Aggregator, and here’s the output after running the program.

Send Coins or Objects

Finally, the interesting part is that you’ll learn to send transactions on the blockchain.

Let’s send some $FUD tokens to another wallet. This works for any coins on the Sui Network.

import {getFullnodeUrl, SuiClient} from '@mysten/sui/client';
import {Ed25519Keypair} from '@mysten/sui/keypairs/ed25519';
import {Transaction} from '@mysten/sui/transactions';

// Set the RPC URL to connect to the Sui mainnet
const rpcUrl = getFullnodeUrl("mainnet");

const client = new SuiClient({url: rpcUrl});

// Create the keypair using the private key
const keypair = Ed25519Keypair.fromSecretKey("suiprivkey1qq9r6rkysny207t5vr7m5025swh7w0wzra9p0553paprhn8zshqsx2rz64r");

// FUD coin type
const FUD_TYPE = '0x76cb819b01abed502bee8a702b4c2d547532c12f25001c9dea795a5e631c26f1::fud::FUD';

async function sendFUD() {
    const tx = new Transaction();

    // Fetch FUD coins owned by the sender
    const coins = await client.getCoins({owner: keypair.getPublicKey().toSuiAddress(), coinType: FUD_TYPE});

    if (coins.data.length === 0) {
        console.log("No FUD coins found in the wallet.");
        return;
    }

    // Choose the first available FUD coin and split it for the transfer (adjust amount if needed)
    const [coin] = tx.splitCoins(coins.data[0].coinObjectId, [100000000]); 

    tx.transferObjects([coin], '0xb0042cf2c5a16d0a240fc1391d570cd5fe06548f860583f1878c327db70f2a22');

    const result = await client.signAndExecuteTransaction({signer: keypair, transaction: tx});
    await client.waitForTransaction({digest: result.digest});

    console.log("Transaction successful. Digest:", result.digest);
}

sendFUD().then(console.log).catch(console.error);

First, I checked that the wallet had some $FUD and split it for the transfer. The tx.transferObjects transfer the split coin to the specified address.

Finally, you need to sign the transaction with the client.signAndExecuteTransaction, and you can wait for the transaction with waitForTransaction to confirm the transaction went through

Conclusion

You’ve learned to interact with the Sui blockchain using the official TypeScript SDK. There’s so much you can build on Sui with your newly acquired knowledge, such as building wallets and bots.

You can take this further by learning how to interact with Move contracts on Sui to build more sophisticated dApps

SPOTLIGHT: A GENTLE INTRODUCTION TO MACHINE LEARNING CONCEPTS IN PYTHON

Ukeje Chukwuemeriwo Goodness — Wed, 05 Jan 2022 11:24:37 +0000

Machine Learning is a branch of Artificial intelligence that deals with the study of computational algorithms and statistical models to perform tasks through patterns and interference instead of explicit tasks. The machine (Your computer) takes in the data and algorithm, learns from it, and then could use it to predict for other new instances. Machine learning is a set of tools used to build models on data that could help predict new types of the same data. Machine learning models are now very popular everywhere. You must have once wondered how Facebook knew that you know someone, or how Spotify got that really cool jam recommended to you. Well, that is machine learning and the same way Facebook could pop up someone you do not know is based on the fact that machine-learning models are not 100% accurate all the time.

Machine Learning comprises of supervised learning where we know the target or past answer and unsupervised learning where there are no targets.

In this article, we will be covering the basics of machine learning by viewing the basic ways machine learning models carry out operations to give us juicy feedback. This article is on supervised learning, as this is an introduction.

The Concepts below cover the basics of machine learning:

GROUPING DATA INTO FEATURES AND TARGETS:

As highlighted earlier, in supervised learning, we aim at predicting values based on past data (target) hence; the dataset with which we work comes with a column, which contains the values, which we are trying to predict. This is the target column. Whilst other columns, which are going to be used to predict the target are called the features. Datasets could be ambiguous, containing useless data, which we would want to get rid of, and here, data analysis and manipulation comes in handy. We would also want to have a good knowledge of the features and we can do this with a good knowledge of exploratory data analysis.
Here is a link to an article on exploratory data analysis article you might find useful.

BASIC MACHINE LEARNING ALGORITHM OPERATIONS

Classification: In classification, machine learning models group data into different parts based on the algorithm provided. Popular classification algorithms include the K Nearest Neighbors, Support Vector Machines amongst others and with the help of a concept called cross-validation; we will be able to pick the best one to work with on our data.
Regression: Regression algorithms operate by giving out the relationship between two or more features in our model. Examples are linear and logistic regression algorithms.
Regression and classification are categorized under the same umbrella of supervised machine learning. … The main difference between them is that the output variable in regression is numerical (or continuous) while that for classification is categorical (or discrete)

THE MACHINE LEARNING WORKFLOW

Importing: By importing, we get the necessary tools we are going to be using on our machine learning model examples are the algorithms and the tools we use for exploratory data analysis

from sklearn.linearmodel import LinearRegression

Instantiating: This is the process of the creation of an instance of the machine learning method. While some of them accept parameters like the k nearest neighbours, others do not accept parameters.

my_model = LinearRegression()

Splitting into training and testing data: We split the data into training and testing sets. We go-ahead to work with the training set and then compare them with the testing set to have a glance at how well our model performed.

from sklearn.model_selection import train_test_split
X = feature columns
y = target column
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

*train_test_split * does a tuple unpacking of the data. The argument as seen above "test_size" is the portion of the data we are willing to allocate to the machine learning method for training.

Fit: By fitting a model, we are feeding the training set of our data to the algorithm to operate on. We could tune how the algorithm operates of the fitted data. We would fit the training data to the machine learning method.

my_model.fit(X_train, y_train)

Predict: The goal of using a machine learning algorithm is to be able to get predictions and feedbacks off of it and we do this using the predict method of the machine learning algorithm. Most machine learning algorithms are already sophisticated and ready to use so most of the work a data scientist does is with refining the data for the algorithm. We predict on the X_test from the train_test_split tuple unpacking.

prediction = my_model.predict(X_test)

EVALUATING A MODEL'S PERFORMANCE

Classification report: A classification report is a metric that gives a table of how well the algorithm performed in percentages. The table contains the precision, recall, and the f1 score. The precision column tells us the percentage score of how well the algorithm classified our model accurately. The recall column gives us feedback on how the algorithm classified data that did not belong to a category while the f1 score is a harmonic mean of the precision and recall. A classification report comes in handy for model evaluation, as it is simple to obtain.

from sklearn.metrics import classification_report
print(classification_report(y_test,predictions))

The confusion matrix: The confusion matrix method is used to summarize how the algorithm performed on our data. In a confusion matrix, the rows correspond to values the algorithm predicted while the columns correspond to the known truths (actuals). Values on the diagonal specify where the algorithm correctly classified our data while the others show where the algorithm failed. Using the confusion matrix we can compare how well different algorithms perform on our data and then go ahead to select the algorithm that best suits our data.

from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_test,predictions))

Cross-Validation: Cross-Validation allows us to compare different machine learning methods and get a sense of how well they will work in practice which helps us choose the machine learning algorithm that best suits our data. Cross-validation does this by splitting the data into training and testing sets. Splitting the data into n numbers, the type of cross-validation is the n cross-validation since the number of splits is arbitrary. Cross-validation uses every part of the split data to deliver.

Bias and Variance: Bias is the inability of a machine learning method to capture the true relationship between features. Variance is the difference in fits between different datasets. In machine learning, a good algorithm is one with a low bias, can accurately model the true relationship, and has low variability (it should be able to provide consistently good predictions over different datasets). When there is a case of a high bias and low variance, the model is under-fitted and when there are a low bias and a high variance, the model is over-fit. Trading bias for variance and vice-versa is a Bias Variance Trade-off.

With knowledge of these basic concepts, learning other machine learning methods would be quite easy.
Follow and share this article if you find it useful. Till I get into your mind on this app next time 🐱‍🏍.