DEV Community: Adetoye Anointing

Setting Up A PostgreSQL Database on AWS: A Step-by-Step Guide to Generate a Service Connection String

Adetoye Anointing — Thu, 11 Jan 2024 19:50:34 +0000

If you're reading this, you've probably struggled with how to set up a Postgres database on AWS and generate a connection for usage in your application.

Perhaps you are like me, highly focused on backend development, and need to add the connection in your .env file. You're probably under a lot of mental strain because you just built or are about to build a $16 per month database instance and AWS wants you to utilize 101% of your brain, making it difficult to connect to the database instance.

This article provides a step-by-step guide on how to create, configure, and connect to your database instance on AWS. The knowledge provided can be applied to a wide range of scenarios.

Prerequisite

This guide will walk you through the steps of creating a free database instance on AWS. However, it is important to note that you will incur charges for using it, so it's advisable to delete the instance when you are not actively using it.

If you already know how to create and configure an instance, skip to the connection string generation section for help with connection.

All highlighted fields and sections in the images of this article are considered important to take note of.

Let's Get Started

First, navigate to Amazon AWS and create/log in to your account.

It can be done using the following link : Amazon AWS

Navigate To The Resources For Creating A Database

In the console's home menu, we select the 'RDS' section as it manages resources and instances for relational databases. The abbreviation stands for 'Relational Database Service'.

Select A Creation Method

The selection method is choosing between setting up your instance yourself or letting Amazon's service take the wheel for you.

For this tutorial, we will select the standard create option as it provides more flexibility to customize.

Database Engine Type Selection

In this article, we will be using Postgres as our database engine, so naturally, it should be our top choice.

After selecting the engine type, we will choose a free tier template for the tutorial. This will also prevent us from getting charged heavily in case we forget to delete the resources.

Important Database Setting

Name The DB instance identifier

It is important to note that adding a database instance identifier that is descriptive is very crucial to identify instance and it should be done for your ease

Credential Settings

This simply involves adding the database username and password that should be tied to the instance on creation

It is essential to create a unique password for the database instance instead of relying on the auto-generated password field for easy recall.

Database Instance Configuration

For the final step of the first phase (creation of Database instance), we will add the most important.

You need to scroll down until you see a section labeled Additional configuration.

This section holds the field that will be used to name our database and it is very important because without it there will be no database created and this part is missed a lot.

Once the entire configuration is complete, your billing should match this if you have done everything correctly.

Then proceed to hit the create database button.

It will take approximately 5-10 minutes or less to create. Once completed, we can proceed to the next phase.

Connection String Generation

This is the second phase, and I must acknowledge that it is the trickiest part of it all.

The format for putting together an Amazon AWS Postgres database connection string together is simply :

postgres://<username>:<password>@<database_endpoint>/<database_name>

username = master username
password = initially added password
database_endpoint = endpoint generated by AWS
database_name = name given to database on creation

To begin with, please navigate to the newly created resources. If you're already there, scroll down a bit and you'll find the database endpoint.

This is a crucial component in generating the connection string.

Afterward, click on the Configuration section in the newly created instance

Get the username, password, and database name from the Configuration section when you scroll slightly downward

Now that we have constructed the connection string, can we start working on the database?

"The answer is no."

Before putting the database instance to good use, one additional configuration must be done for full remote access.

Access Configuration

This involves declaring access to the database instance as part of a security measure to prevent unwanted connections and access to resources.

To accomplish this, you will need to add an inbound rule to the database instance.

In the Connectivity & security section, we will click on the link in the VPC security groups field

Let's navigate to the Security group ID of the database instance to make modifications.

Click the edit inbound rules button

Click on Add rules below the already predefined rules and add two rules

highlighted red = protocol type
highlighted blue = the database port [which by default is "5432"]
highlighted cyan = connection source

Save these rules put in place and configured. You are good to connect to your database on Amazon RDS and have fun with it.

Note that the highlighted green indicates a security warning. If possible, please provide the IP address of your service instance or local machine to avoid this security flag that might eventually turn into a security vulnerability.

Conclusion

with this guide, you should be able to set up a Postgres database on Amazon AWS, configure and have it connected to your local machine or web server using the connection string.

I hope this article was insightful and that you find It helpful

Pro tip: Avoid unnecessary charges by shutting down and deleting unused AWS instances.

Thank you for reading, drop a comment if you have a question, and have fun :)

Handling Dynamic API Response In Go

Adetoye Anointing — Mon, 01 Jan 2024 20:29:11 +0000

A Go developer working on a service that requires a third-party API integration and that integration has a webhook or something similar that returns a changing response body structure depending on the event sent to and through the hook.

A lot of people do not essentially know how to handle this scenario in Go at least using the gracious and glorious JSON package to handle something simple yet overly complicated task in Go.

Today, we will have a walkthrough on handling dynamic API responses in Go using the JSON package.

Prerequisite

It is crucial to understand that dynamic API responses take different forms, some are events triggered in response to an activity in the service. Whether it is a FinTech API using a webhook to notify the system of an event or a user triggering a notification to an admin dashboard on a specific event.

These dynamic APIs hold a means of identification that can be used to identify them and separate them into their different predefined structure in most cases.

let's get into the fun part of this article [the coding part]

To get started with this tutorial, you will need:

Go installed and set up
An IDE or test editor
Practical Knowledge of Go
Postman

Initial stage [ Setup your project repo ]

Open the terminal and set up our project repo using the command below. (This of course works on Linux and macOS)

mkdir Documents/dynamic-API && touch Documents/dynamic-API/
main.go && code Documents/dynamic-API

go mod init [your-github-name]/handling-dynamic-api

Create our main function and setup for development.

package main

import "log/slog"


func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

}

Build the function that handles the API response for the service.

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))
    }
}

This function will hold the entire logic that handles the response from the webhook.
Since we are handling this without a third-party library or framework, we will be using the built-in Go http library to handle this bit which will require importing net/http while also defining a logger of type *slog.Logger as an argument to the HandleDynamicAPI function

random fun fact: if you are not a VIM / Nano user, you might not need to worry about importation cause your text editor probably have it handled

Hence; your current code should look something like this

package main

import (
    "log/slog"
    "net/http"
)


func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

}

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))

    }
}

Catch And Identify The Response Event Or Identifier (in case of an API with events or identifiers)

Event Identifier Structure

//this will be used to identify the event type
//
//We care about just the event head
type eventIdentfier struct {
    Event string `json:"event"`
}

Logic To Catch And Identify Event

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))

        var (
            eventIdentfier    eventIdentfier
            jsonData          json.RawMessage
        )

        if err := json.NewDecoder(r.Body).Decode(&jsonData); err != nil {
            l.Error("error decoding json response", "error context", err)
            return
        }

        if err := json.Unmarshal(jsonData, &eventIdentfier); err != nil {
            l.Error("error unmarshalling json data message", "error context", err)
            return
        }
    }
}

What the written code above does is it catches the API response body data and decodes it into the jsonData variable which is a type of json.RawMessage and that jsonData value is unmarshalled into the eventIdentifier variable to be used in the later stage of development.

How the json.RawMessage works are it delays json decoding of the body making it flexible to use for re-unmarshalling as I will call it and in our case used for event identification and eventually decoding into a defined json structure.

Using Event To Identify Execution Case

In this part, we will be going deeper into Go using control flow to execute logic and many cases functions after identification of the event that should be executed.

firstly, we want to know what the event value will be, which will be used to handle the control flow of the program.
These are usually found in the API / webhook documentation and our case we are using paystack.

For information about Paystack's webhooks, check out their API documentation and visit Paystack Webhook URL.

The event we will simulate for this tutorial is :

Payment Request Pending
Payment Request Successful

Looking at the response structure provided by paystack we can easily create a struct and fill out the fields needed but in our case due to some reason unknown to me for the fun of it; I will be using the entire field :).

Response Body Data Structure For Unmarshalling

type paymentPending struct {
    Event string `json:"event"`
    Data  struct {
        ID               int       `json:"id"`
        Domain           string    `json:"domain"`
        Amount           int       `json:"amount"`
        Currency         string    `json:"currency"`
        DueDate          any       `json:"due_date"`
        HasInvoice       bool      `json:"has_invoice"`
        InvoiceNumber    any       `json:"invoice_number"`
        Description      string    `json:"description"`
        PdfURL           any       `json:"pdf_url"`
        LineItems        []any     `json:"line_items"`
        Tax              []any     `json:"tax"`
        RequestCode      string    `json:"request_code"`
        Status           string    `json:"status"`
        Paid             bool      `json:"paid"`
        PaidAt           any       `json:"paid_at"`
        Metadata         any       `json:"metadata"`
        Notifications    []any     `json:"notifications"`
        OfflineReference string    `json:"offline_reference"`
        Customer         int       `json:"customer"`
        CreatedAt        time.Time `json:"created_at"`
    } `json:"data"`
}

type paymentSuccessful struct {
    Event string `json:"event"`
    Data  struct {
        ID            int       `json:"id"`
        Domain        string    `json:"domain"`
        Amount        int       `json:"amount"`
        Currency      string    `json:"currency"`
        DueDate       any       `json:"due_date"`
        HasInvoice    bool      `json:"has_invoice"`
        InvoiceNumber any       `json:"invoice_number"`
        Description   string    `json:"description"`
        PdfURL        any       `json:"pdf_url"`
        LineItems     []any     `json:"line_items"`
        Tax           []any     `json:"tax"`
        RequestCode   string    `json:"request_code"`
        Status        string    `json:"status"`
        Paid          bool      `json:"paid"`
        PaidAt        time.Time `json:"paid_at"`
        Metadata      any       `json:"metadata"`
        Notifications []struct {
            SentAt  time.Time `json:"sent_at"`
            Channel string    `json:"channel"`
        } `json:"notifications"`
        OfflineReference string    `json:"offline_reference"`
        Customer         int       `json:"customer"`
        CreatedAt        time.Time `json:"created_at"`
    } `json:"data"`
}

declare a variable of both defined structures in the existing variable list

paymentPending paymentPending paymentSuccessful paymentSuccessful

After defining the struct and declaring the variables, we will then proceed to write out the control flow and implement the logic for the program

        switch eventIdentfier.Event {
        case "paymentrequest.pending":
            l.Info("payment pending hook event", "response event title", eventIdentfier.Event)

            if err := json.Unmarshal(jsonData, &paymentPending); err != nil {
                l.Error("error marshalling pending payment data", "error context", err)
                return
            }

            if err := json.NewEncoder(w).Encode(map[string]any{"event type": paymentPending.Event, "amount": paymentPending.Data.Amount}); err != nil {
                l.Error("error encoding data to send as response", "error context", err)
                return
            }
            w.Header().Add("Content-Type", "application/json")

            l.Info("pending response data unmarshalled successfully", "pending ID", paymentPending.Data.ID, "pending amount", paymentPending.Data.Amount)
        case "paymentrequest.success":
            l.Info("payment successful hook event", "response event title", eventIdentfier.Event)

            if err := json.Unmarshal(jsonData, &paymentSuccessful); err != nil {
                l.Error("error marshalling successful payment data", "error context", err)
                return
            }

            if err := json.NewEncoder(w).Encode(map[string]string{"event type": paymentSuccessful.Event}); err != nil {
                return
            }
            w.Header().Add("Content-Type", "application/json")

            l.Info("success response data unmarshalled successfully", "success ID", paymentSuccessful.Data.ID, "success amount", paymentSuccessful.Data.Amount)
        default:
            l.Info("no event type found", "response event title", eventIdentfier.Event)
            w.WriteHeader(http.StatusInternalServerError)
        }

The code above retrieves the event from the variable named eventIdentifier. It then uses the switch control flow to determine which API event has been passed to the endpoint. After identifying the API event, the data is decoded into the defined struct. Finally, an API response is returned with the required specified data.

There are instances where data is handled differently, and it doesn't involve returning a few selected fields as shown in the example above. Such cases include:

Storing the data in a database
Using the data for manipulation purposes
Utilizing the data for verifying a triggered action, and so on.

focusing on the basics is crucial for building a strong foundation to tackle more advanced projects.

Plug In The Function To An Endpoint

Plugging a function into an endpoint is easily one of the first you learn while studying Go and I am counting that this won't be the tricky part.

Plug in the function to the http handler using this line

http.HandleFunc("/dynamic-hook", HandleDynamicAPI(logger))

setup service listener

go log.Fatal(http.ListenAndServe(":3000", nil))

These should be done in the main function, hence; the main function should look like this

func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

    http.HandleFunc("/dynamic-hook", HandleDynamicAPI(logger))

    go log.Fatal(http.ListenAndServe(":3000", nil))
}

Running, Testing And Finishing

Run Go Server

We run the program with our gracious Go command in the terminal:

go run main.go

Test Endpoint And Simulate Webhook

To do this we have to copy the webhook json data and pass it to the designated endpoint using Postman

paymentrequest.success event response
paymentrequest.pending event response

Conclusion

You should have a basic understanding of how to handle dynamic API response bodies, including use cases and the importance of the json package in Go for handling responses.

I hope you found this tutorial helpful.

Have fun practicing :)

Link To The Entire Code Sample

Handling Dynamic API Response