Adetoye Anointing

Posted on Jan 1

Handling Dynamic API Response In Go

#go #json #api #backend

A Go developer working on a service that requires a third-party API integration and that integration has a webhook or something similar that returns a changing response body structure depending on the event sent to and through the hook.

A lot of people do not essentially know how to handle this scenario in Go at least using the gracious and glorious JSON package to handle something simple yet overly complicated task in Go.

Today, we will have a walkthrough on handling dynamic API responses in Go using the JSON package.

Prerequisite

It is crucial to understand that dynamic API responses take different forms, some are events triggered in response to an activity in the service. Whether it is a FinTech API using a webhook to notify the system of an event or a user triggering a notification to an admin dashboard on a specific event.

These dynamic APIs hold a means of identification that can be used to identify them and separate them into their different predefined structure in most cases.

let's get into the fun part of this article [the coding part]

To get started with this tutorial, you will need:

Go installed and set up
An IDE or test editor
Practical Knowledge of Go
Postman

Initial stage [ Setup your project repo ]

Open the terminal and set up our project repo using the command below. (This of course works on Linux and macOS)

mkdir Documents/dynamic-API && touch Documents/dynamic-API/
main.go && code Documents/dynamic-API

go mod init [your-github-name]/handling-dynamic-api

Create our main function and setup for development.

package main

import "log/slog"


func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

}

Build the function that handles the API response for the service.

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))
    }
}

This function will hold the entire logic that handles the response from the webhook.
Since we are handling this without a third-party library or framework, we will be using the built-in Go http library to handle this bit which will require importing net/http while also defining a logger of type *slog.Logger as an argument to the HandleDynamicAPI function

random fun fact: if you are not a VIM / Nano user, you might not need to worry about importation cause your text editor probably have it handled

Hence; your current code should look something like this

package main

import (
    "log/slog"
    "net/http"
)


func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

}

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))

    }
}

Catch And Identify The Response Event Or Identifier (in case of an API with events or identifiers)

Event Identifier Structure

//this will be used to identify the event type
//
//We care about just the event head
type eventIdentfier struct {
    Event string `json:"event"`
}

Logic To Catch And Identify Event

func HandleDynamicAPI(l *slog.Logger) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        l.Info("This API is connected", "user", os.Getenv("USER"))

        var (
            eventIdentfier    eventIdentfier
            jsonData          json.RawMessage
        )

        if err := json.NewDecoder(r.Body).Decode(&jsonData); err != nil {
            l.Error("error decoding json response", "error context", err)
            return
        }

        if err := json.Unmarshal(jsonData, &eventIdentfier); err != nil {
            l.Error("error unmarshalling json data message", "error context", err)
            return
        }
    }
}

What the written code above does is it catches the API response body data and decodes it into the jsonData variable which is a type of json.RawMessage and that jsonData value is unmarshalled into the eventIdentifier variable to be used in the later stage of development.

How the json.RawMessage works are it delays json decoding of the body making it flexible to use for re-unmarshalling as I will call it and in our case used for event identification and eventually decoding into a defined json structure.

Using Event To Identify Execution Case

In this part, we will be going deeper into Go using control flow to execute logic and many cases functions after identification of the event that should be executed.

firstly, we want to know what the event value will be, which will be used to handle the control flow of the program.
These are usually found in the API / webhook documentation and our case we are using paystack.

For information about Paystack's webhooks, check out their API documentation and visit Paystack Webhook URL.

The event we will simulate for this tutorial is :

Payment Request Pending
Payment Request Successful

Looking at the response structure provided by paystack we can easily create a struct and fill out the fields needed but in our case due to some reason unknown to me for the fun of it; I will be using the entire field :).

Response Body Data Structure For Unmarshalling

type paymentPending struct {
    Event string `json:"event"`
    Data  struct {
        ID               int       `json:"id"`
        Domain           string    `json:"domain"`
        Amount           int       `json:"amount"`
        Currency         string    `json:"currency"`
        DueDate          any       `json:"due_date"`
        HasInvoice       bool      `json:"has_invoice"`
        InvoiceNumber    any       `json:"invoice_number"`
        Description      string    `json:"description"`
        PdfURL           any       `json:"pdf_url"`
        LineItems        []any     `json:"line_items"`
        Tax              []any     `json:"tax"`
        RequestCode      string    `json:"request_code"`
        Status           string    `json:"status"`
        Paid             bool      `json:"paid"`
        PaidAt           any       `json:"paid_at"`
        Metadata         any       `json:"metadata"`
        Notifications    []any     `json:"notifications"`
        OfflineReference string    `json:"offline_reference"`
        Customer         int       `json:"customer"`
        CreatedAt        time.Time `json:"created_at"`
    } `json:"data"`
}

type paymentSuccessful struct {
    Event string `json:"event"`
    Data  struct {
        ID            int       `json:"id"`
        Domain        string    `json:"domain"`
        Amount        int       `json:"amount"`
        Currency      string    `json:"currency"`
        DueDate       any       `json:"due_date"`
        HasInvoice    bool      `json:"has_invoice"`
        InvoiceNumber any       `json:"invoice_number"`
        Description   string    `json:"description"`
        PdfURL        any       `json:"pdf_url"`
        LineItems     []any     `json:"line_items"`
        Tax           []any     `json:"tax"`
        RequestCode   string    `json:"request_code"`
        Status        string    `json:"status"`
        Paid          bool      `json:"paid"`
        PaidAt        time.Time `json:"paid_at"`
        Metadata      any       `json:"metadata"`
        Notifications []struct {
            SentAt  time.Time `json:"sent_at"`
            Channel string    `json:"channel"`
        } `json:"notifications"`
        OfflineReference string    `json:"offline_reference"`
        Customer         int       `json:"customer"`
        CreatedAt        time.Time `json:"created_at"`
    } `json:"data"`
}

declare a variable of both defined structures in the existing variable list

paymentPending paymentPending paymentSuccessful paymentSuccessful

After defining the struct and declaring the variables, we will then proceed to write out the control flow and implement the logic for the program

        switch eventIdentfier.Event {
        case "paymentrequest.pending":
            l.Info("payment pending hook event", "response event title", eventIdentfier.Event)

            if err := json.Unmarshal(jsonData, &paymentPending); err != nil {
                l.Error("error marshalling pending payment data", "error context", err)
                return
            }

            if err := json.NewEncoder(w).Encode(map[string]any{"event type": paymentPending.Event, "amount": paymentPending.Data.Amount}); err != nil {
                l.Error("error encoding data to send as response", "error context", err)
                return
            }
            w.Header().Add("Content-Type", "application/json")

            l.Info("pending response data unmarshalled successfully", "pending ID", paymentPending.Data.ID, "pending amount", paymentPending.Data.Amount)
        case "paymentrequest.success":
            l.Info("payment successful hook event", "response event title", eventIdentfier.Event)

            if err := json.Unmarshal(jsonData, &paymentSuccessful); err != nil {
                l.Error("error marshalling successful payment data", "error context", err)
                return
            }

            if err := json.NewEncoder(w).Encode(map[string]string{"event type": paymentSuccessful.Event}); err != nil {
                return
            }
            w.Header().Add("Content-Type", "application/json")

            l.Info("success response data unmarshalled successfully", "success ID", paymentSuccessful.Data.ID, "success amount", paymentSuccessful.Data.Amount)
        default:
            l.Info("no event type found", "response event title", eventIdentfier.Event)
            w.WriteHeader(http.StatusInternalServerError)
        }

The code above retrieves the event from the variable named eventIdentifier. It then uses the switch control flow to determine which API event has been passed to the endpoint. After identifying the API event, the data is decoded into the defined struct. Finally, an API response is returned with the required specified data.

There are instances where data is handled differently, and it doesn't involve returning a few selected fields as shown in the example above. Such cases include:

Storing the data in a database
Using the data for manipulation purposes
Utilizing the data for verifying a triggered action, and so on.

focusing on the basics is crucial for building a strong foundation to tackle more advanced projects.

Plug In The Function To An Endpoint

Plugging a function into an endpoint is easily one of the first you learn while studying Go and I am counting that this won't be the tricky part.

Plug in the function to the http handler using this line

http.HandleFunc("/dynamic-hook", HandleDynamicAPI(logger))

setup service listener

go log.Fatal(http.ListenAndServe(":3000", nil))

These should be done in the main function, hence; the main function should look like this

func main() {
    // setup a logger using slog
    logger := slog.New(slog.NewTextHandler(os.Stdout, nil))

    logger.Info("Hello Terminal 👋", "user", os.Getenv("USER"))

    http.HandleFunc("/dynamic-hook", HandleDynamicAPI(logger))

    go log.Fatal(http.ListenAndServe(":3000", nil))
}

Running, Testing And Finishing

Run Go Server

We run the program with our gracious Go command in the terminal:

go run main.go

Test Endpoint And Simulate Webhook

To do this we have to copy the webhook json data and pass it to the designated endpoint using Postman

paymentrequest.success event response
paymentrequest.pending event response

Conclusion

You should have a basic understanding of how to handle dynamic API response bodies, including use cases and the importance of the json package in Go for handling responses.

I hope you found this tutorial helpful.

Have fun practicing :)