DEV Community: Ramu Mangalarapu

Working with 2D slices in Golang

Ramu Mangalarapu — Mon, 15 Aug 2022 13:15:40 +0000

Have you ever passed 2D slices as function argument in Golang. 2D slice is things with slices of slices.


package main

import (
    "fmt"
)

func print2DSlice(a [][]int) {
    // let us iterate over 2D slice
    for i := 0; i < len(a); i++ {
        oD := a[i] // access 0th array in 2D slice
        for j := 0; j < len(oD); j++ {
            fmt.Printf("%d ", oD[j])
        }
        fmt.Println()
    }
    return
}

func main() {
    tDSlice := make([][]int, 0)
        // or tDSlice:=[][]int{}
    tDSlice = append(tDSlice, []int{1, 2, 3, 4, 5})
    tDSlice = append(tDSlice, []int{12, 32, 43, 423, 52})
    tDSlice = append(tDSlice, []int{2341, 322, 323, 324, 53})
    tDSlice = append(tDSlice, []int{3321, 2423, 33232, 432, 532})
    tDSlice = append(tDSlice, []int{14343, 24343, 34343, 44343, 54343})
    tDSlice = append(tDSlice, []int{14343, 24343, 3434334, 4434, 54343})
    tDSlice = append(tDSlice, []int{4343431, 43432, 34343, 44343, 4343435})

    print2DSlice(tDSlice)
}

Ref: https://www.dotnetperls.com/2d-go
Thank you.

Let us try to understand what is docker and a simple api in Golang

Ramu Mangalarapu — Sun, 07 Aug 2022 09:31:00 +0000

Docker is an open source containerization platform. It enables developers to package applications into containers—standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment -IBM

So, what is containerization then ?

Containerization is a form of virtualization where applications run in isolated user spaces, called containers, while using the same shared operating system (OS). One of the benefits of containerization is that a container is essentially a fully packaged and portable computing environment. -Citrix

And also
From Docker about container

History of Container

keep surfing to understand the history of computing, it is pretty much interesting!!
After all it is essential to understand the core part of any software concept in the IT and services industry by asking what is it ? why is it ? how is it ? :)

Bare metal to serverless

Overview of Servers

We know that we can have linux container and windows container
(I really not sure about Apple Mac OS X container :) )

If you have bit of application development experience, just assume that container is mini operating system but only bundled up with specific deps, libs etc.. to run the code you have written. You cannot use the container like full operating system. We build the containers based on our requirements to run our software. And finally docker is containerization platform, docker is not the only containerization platform, there are other platforms also [Google it]. And again docker and container both are different.

Anyway, let us say you want to build and run a code for fetching all the opened restaurants near by you (of course you provide place as input to the code), you can write code, once it is working fine in your local, you could just write simple docker commands such as how you want to build and run your code in the mini operating system.

Ok, I think it is too much about container and docker. Let us write simple Golang application, container it and run it. [We can even push to our docker hub which is just like git VCS but for docker images].

go mod init github.com/YOUR_GITHUB_USERNAME/go-docker-example

module github.com/YOUR_GITHUB_USERNAME/go-docker-example

go 1.18

in case if you add deps, just tidy it but for this tutorial, we are not using any packages

go mod tidy

package main

import (
    "fmt"
    "html"
    "log"
    "net/http"
)

func main() {
    // index handler
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
    })

    // hello handler
    http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintf(w, "Hello")
    })

    // starting the server
    log.Fatal(http.ListenAndServe(":8081", nil))
}

Let us write the Dockerfile which are just commands to build our mini OS from base image.

Where we write Dockerfile, we build it, then we get image and with the image, when we run it, we get container.

# Let us specify base image required to for our application
# usually we specify base images to build application
# just assume that we will get linux OS with golang installed in it
# to build and run go programs
FROM golang:1.18-alpine

# all the operations we do here are linux commands only
# basically it is like just you have some linux OS,
# the things you specify here just commands to fetch source code
# and run it

# create new directory app where 
# we we build our application
RUN mkdir /app

# add everything from current source to this container
ADD . /app

# from now onwards this would be our workdire where we would
# be performing all the operations
WORKDIR /app

## let us run go mod download command to pull in any dependencies
RUN go mod download

## let us now build 
RUN go build -o main .

## start command which kicks off
## this newly created binary executable
CMD ["/app/main"]

Repo: https://github.com/developer1622/go-docker-example

Thank you, Happy learning. More to share!

for{ 
  unlearn
  learn
  relearn 
}

Building REST APIs in Golang go-gin with persistence database Postgres

Ramu Mangalarapu — Sun, 20 Mar 2022 11:20:24 +0000

Let's build REST APIs in Go using Gin web framework.

I will adding authentication, docker image and sample deployment in Google cloud in next coming articles.

We are going to build APIs around Albums that I took from go.dev but I will extend by using the Postgres database. So, this will have persistent data storage.

Usually when we are working with web frameworks, they usually contains all the required components to build full application without importing any other extra library. See here

We will be having routing, JSON/XML parsers (we could say famous data interchange format parsers), validations, sanitizations of input data, HTTP clients to make call to external APIs, database drivers, things that makes response writing easy to server, ORMs (object relational mapper), Security measures such as CSRF, XSS, SQL Injections, logging etc.. which are required to build application easily.

Just imagine, we do not have fully tested router functionality and in case if want to build router from scratch, it is really hard for normal developer to build and maintain all the times. All the libraries, frameworks and SDKs provide abstraction for us to build application easily. We only need to worry about what is our business logic. In fact there are even Go libraries that generated REST APIs skelton for us. Awesome right ?

Yes, it is, it is happening and it will continue to happen that we do not bother to write code for common web application items such as authentication, authorization, input validations, response generation, unit testing, integration testing, logging, middleware, take any functionality you wanted to have, there will be multiple libraries and APIs (this could be SaaS companies which provide enterprise level things) which makes things easy for us.

I think we are going beyond our current topic, all I am sharing here is basic example of building REST APIs which has persistent memory with Postgres Server.

Let's initialize go module to create to rest apis.

go mod init example.com/web-services

First of all we need to connect to database, I am keeping all the files in same main package as this is simple one but usually we should divide into packages, there are no official project layout for Go but usually developers follow things like handlers, config, services, apis, domain, etc like that, but there are like golang project layout

postgressql.sql


CREATE TABLE IF NOT EXISTS artist (
    id SERIAL PRIMARY KEY,
    name VARCHAR ( 50 ) NOT NULL --this is name of the Author,
    title VARCHAR ( 50 ) UNIQUE NOT NULL,
    price VARCHAR ( 255 ) NOT NULL
);

SELECT id,name,title,price FROM artist LIMIT 5 OFFSET 1;

UPDATE artist 
SET name='nae56',title='tle56', price=66.7667
WHERE id=8;

SELECT * FROM artist;

SELECT * FROM artist WHERE id=8;

DELETE FROM artist WHERE id=1;

INSERT INTO artist(name,title,price) VALUES('name3','title3','price3') ON CONFLICT DO NOTHING;

database.go


package main

import (
    "database/sql"
    "fmt"
    "log"

    _ "github.com/lib/pq"
)

// Env holds database connection to Postgres
type Env struct {
    DB *sql.DB
}

// database variables
// usually we should get them from env like os.Getenv("variableName")
const (
    host     = "localhost"
    port     = 5439
    user     = "postgres"
    password = "root"
    dbname   = "artists"
)

// ConnectDB tries to connect DB and on succcesful it returns
// DB connection string and nil error, otherwise return empty DB and the corresponding error.
func ConnectDB() (*sql.DB, error) {
    connString := fmt.Sprintf("host=%s port=%d user=%s password=%s dbname = %s sslmode=disable", host, port, user, password, dbname)
    db, err := sql.Open("postgres", connString)
    if err != nil {
        log.Printf("failed to connect to database: %v", err)
        return &sql.DB{}, err
    }
    return db, nil
}

main.go

package main

import (
    "log"

    "github.com/gin-gonic/gin"
)

func main() {
    // connect to DB first
    env := new(Env)
    var err error
    env.DB, err = ConnectDB()
    if err != nil {
        log.Fatalf("failed to start the server: %v", err)
    }

    router := gin.Default()
    router.GET("/albums/:id", env.GetAlbumByID)
    router.GET("/albums", env.GetAlbums)
    router.POST("/albums", env.PostAlbum)
    router.PUT("/albums", env.UpdateAlbum)
    router.DELETE("/albums/:id", env.DeleteAlbumByID)

    router.Run("localhost:8080")
}

handler.go

package main

// This is to ensure we do not overload the server when we have millions of rows
var recordFetchLimit = 100 // testing purpose

album_get.go

package main

import (
    "database/sql"
    "fmt"
    "log"
    "net/http"
    "strconv"

    "github.com/gin-gonic/gin"

    _ "github.com/lib/pq"
)

// GetAlbumByID locates the album whose ID value matches the id
// parameter sent by the client, then returns that album as a response.
func (env Env) GetAlbumByID(c *gin.Context) {
    id, err := strconv.Atoi(c.Param("id"))
    if err != nil {
        e := fmt.Sprintf("received invalid id path param which is not string: %v", c.Param("id"))
        log.Println(e)
        makeGinResponse(c, http.StatusNotFound, e)
        return
    }

    var name, title string
    var price float64
    q := `SELECT * FROM artist WHERE id=$1;`
    row := env.DB.QueryRow(q, id)
    err = row.Scan(&id, &name, &title, &price)
    switch err {
    case sql.ErrNoRows:
        log.Printf("no rows are present for alubum with id: %d", id)
        makeGinResponse(c, http.StatusBadRequest, err.Error())
    case nil:
        log.Printf("we are able to fetch album with given id: %d", id)
        c.JSON(http.StatusOK, NewAlbum(id, title, name, price))
    default:
        e := fmt.Sprintf("error: %v occurred while reading the databse for Album record with id: %d", err, id)
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, err.Error())
    }
}

// GetAlbums responds with the list of all albums as JSON.
func (env Env) GetAlbums(c *gin.Context) {
    // Note:
    //
    // pagnination can be impleted in may ways, but I am following one of the way,
    // if you feel this is confusing, please read medium article that I have added below
    // For this page and perPage isseus, front end engineers can take care of it
    // by escaping and sanitization of query params.
    // Please see: https://www.enterprisedb.com/postgres-tutorials/how-use-limit-and-offset-postgresql
    // Please see: https://levelup.gitconnected.com/creating-a-data-pagination-function-in-postgresql-2a032084af54
    page := c.Query("page") // AKA limit in SQL terms
    if page == "" {
        e := "missing query param: page"
        log.Println(e)
        makeGinResponse(c, http.StatusNotFound, e)
        return
    }

    perPage := c.Query("perPage") // AKA limit in SQL terms
    if perPage == "" {
        e := "missing query param: perPage"
        log.Println(e)
        makeGinResponse(c, http.StatusNotFound, e)
        return
    }

    limit, err := strconv.Atoi(page)
    if err != nil {
        e := fmt.Sprintf("received invalid page query param which is not integer : %v", page)
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    if limit > recordFetchLimit {
        // Seems some bad user or front end developer playing with query params!
        e := fmt.Sprintf("we agreed to fetch less than %d records but we received request for %d", recordFetchLimit, limit)
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    offset, err := strconv.Atoi(perPage)
    if err != nil {
        e := fmt.Sprintf("received invalid offset query param which is not integer : %v", page)
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    // anyway, let's check if offset is a negative value
    if offset < 0 {
        e := "offset query param cannot be negative"
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    q := `SELECT id,name,title,price FROM artist LIMIT $1 OFFSET $2;`
    rows, err := env.DB.Query(q, limit, offset)
    switch err {
    case sql.ErrNoRows:
        defer rows.Close()
        e := "no rows records found in artist table to read"
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
    case nil:
        defer rows.Close()
        a := make([]Album, 0)
        var rowsReadErr bool
        for rows.Next() {
            var id int
            var name, title string
            var price float64
            err = rows.Scan(&id, &name, &title, &price)
            if err != nil {
                log.Printf("error occurred while reading the database rows: %v", err)
                rowsReadErr = true
                break
            }
            a = append(a, NewAlbum(id, title, name, price))
        }

        if rowsReadErr {
            log.Println("we are not able to fetch few records")
        }

        // let's return the read rows at least
        log.Printf("we are able to fetch albums for requested limit: %d and offest: %d", limit, offset)
        c.JSON(http.StatusOK, a)
    default:
        defer rows.Close()
        // this should not happen
        e := "some internal database server error"
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
    }
}

album_post.go

package main

import (
    "log"
    "net/http"

    "github.com/gin-gonic/gin"

    _ "github.com/lib/pq"
)

// PostAlbums adds an album from JSON received in the request body.
func (env Env) PostAlbum(c *gin.Context) {
    // Call BindJSON to bind the received JSON to
    // newAlbum.
    var newAlbum Album
    if err := c.BindJSON(&newAlbum); err != nil {
        log.Printf("invalid JSON body: %v", err)
        makeGinResponse(c, http.StatusNotFound, err.Error())
        return
    }

    q := `INSERT INTO artist(name,title,price) VALUES($1,$2,$3) ON CONFLICT DO NOTHING`
    result, err := env.DB.Exec(q, newAlbum.Artist, newAlbum.Title, newAlbum.Price)
    if err != nil {
        log.Printf("error occurred while inserting new record into artist table: %v", err)
        makeGinResponse(c, http.StatusInternalServerError, err.Error())
        return
    }

    // checking the number of rows affected
    n, err := result.RowsAffected()
    if err != nil {
        log.Printf("error occurred while checking the returned result from database after insertion: %v", err)
        makeGinResponse(c, http.StatusInternalServerError, err.Error())
        return
    }

    // if no record was inserted, let us say client has failed
    if n == 0 {
        e := "could not insert the record, please try again after sometime"
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
        return
    }

    // NOTE:
    //
    // Here I wanted to return the location for newly created Album but this
    // 'pq' library does not support, LastInsertionID functionality.
    m := "successfully created the record"
    log.Println(m)
    makeGinResponse(c, http.StatusOK, m)
}

album_put.go

package main

import (
    "fmt"
    "log"
    "net/http"

    "github.com/gin-gonic/gin"

    _ "github.com/lib/pq"
)

// UpdateAlbum updates the Album with the given details if record found.
func (env Env) UpdateAlbum(c *gin.Context) {
    // Call BindJSON to bind the received JSON to
    // toBeUpdatedAlbum.
    var toBeUpdatedAlbum Album
    if err := c.BindJSON(&toBeUpdatedAlbum); err != nil {
        e := fmt.Sprintf("invalid JSON body: %v", err)
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    q := `UPDATE artist 
    SET name=$1,title=$2, price=$3
    WHERE id=$4;`
    result, err := env.DB.Exec(q, toBeUpdatedAlbum.Artist, toBeUpdatedAlbum.Title, toBeUpdatedAlbum.Price, toBeUpdatedAlbum.ID)
    if err != nil {
        e := fmt.Sprintf("error: %v occurred while updating artist record with id: %d", err, toBeUpdatedAlbum.ID)
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
        return
    }

    // checking the number of rows affected
    n, err := result.RowsAffected()
    if err != nil {
        e := fmt.Sprintf("error occurred while checking the returned result from database after updation: %v", err)
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
    }

    // if no record was updated, let us say client has failed
    if n == 0 {
        e := "could not update the record, please try again after sometime"
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
        return
    }

    m := "successfully updated the record"
    log.Println(m)
    makeGinResponse(c, http.StatusOK, m)
}

album_delete.go

package main

import (
    "fmt"
    "log"
    "net/http"
    "strconv"

    "github.com/gin-gonic/gin"

    _ "github.com/lib/pq"
)

// DeleteAlbumByID locates the album whose ID value matches the id
// parameter sent by the client, then returns that corresponding message.
func (env Env) DeleteAlbumByID(c *gin.Context) {
    id, err := strconv.Atoi(c.Param("id"))
    if err != nil {
        e := fmt.Sprintf("received invalid id path param which is not string: %v", c.Param("id"))
        log.Println(e)
        makeGinResponse(c, http.StatusNotFound, e)
        return
    }

    q := `DELETE FROM artist WHERE id = $1;`
    result, err := env.DB.Exec(q, id)
    if err != nil {
        e := fmt.Sprintf("error occurred while deleting artist record with id: %d and error is: %v", id, err)
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
        return
    }

    // checking the number of rows affected
    n, err := result.RowsAffected()
    if err != nil {
        e := fmt.Sprintf("error occurred while checking the returned result from database after deletion: %v", err)
        log.Println(e)
        makeGinResponse(c, http.StatusInternalServerError, e)
        return
    }

    // if no record was deleted, let us inform that there might be no
    // records to delete for this given album ID.
    if n == 0 {
        e := "could not delete the record, there might be no records for the given ID"
        log.Println(e)
        makeGinResponse(c, http.StatusBadRequest, e)
        return
    }

    m := "successfully deleted the record"
    log.Println(m)
    makeGinResponse(c, http.StatusOK, m)
}

We can add business object in domain.go

package main

// Album holds of few important details about it.
type Album struct {
    ID     int     `json:"id,omitempty"`
    Title  string  `json:"title"`
    Artist string  `json:"artist"`
    Price  float64 `json:"price"`
}

// NewAlbum is Album constructor.
func NewAlbum(id int, title, artist string, price float64) Album {
    return Album{id, title, artist, price}
}

There are certain things in application we repeat again, may be we can make one function (DRY principal) add it in the utils.go

package main

import (
    "github.com/gin-gonic/gin"
)

func makeGinResponse(c *gin.Context, statusCode int, value string) {
    c.JSON(statusCode, gin.H{
        "message":    value,
        "statusCode": statusCode,
    })
}

In case if you want try to just take all the codes and copy it run go mod download or go mod tidy to get go.sum and go.mod files.

I have tested them fully, please find Postman collection

postmainCollection.json


{
    "info": {
        "_postman_id": "b7f050de-a273-408c-b11d-3ec27fb8c5d4",
        "name": "Learning01",
        "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
    },
    "item": [
        {
            "name": "Get Albums",
            "request": {
                "method": "GET",
                "header": [],
                "url": {
                    "raw": "localhost:8080/albums?page=100&perPage=78",
                    "host": [
                        "localhost"
                    ],
                    "port": "8080",
                    "path": [
                        "albums"
                    ],
                    "query": [
                        {
                            "key": "page",
                            "value": "100"
                        },
                        {
                            "key": "perPage",
                            "value": "78"
                        }
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Get Albums By ID",
            "request": {
                "method": "GET",
                "header": [],
                "url": {
                    "raw": "localhost:8080/albums/10",
                    "host": [
                        "localhost"
                    ],
                    "port": "8080",
                    "path": [
                        "albums",
                        "10"
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Post Album",
            "request": {
                "method": "POST",
                "header": [],
                "body": {
                    "mode": "raw",
                    "raw": "{\r\n    \"id\":6667778,\r\n    \"title\":\"newTitle\",\r\n    \"artist\":\"dfjkfdkj\",\r\n    \"price\":123434.49\r\n}",
                    "options": {
                        "raw": {
                            "language": "json"
                        }
                    }
                },
                "url": {
                    "raw": "localhost:8080/albums",
                    "host": [
                        "localhost"
                    ],
                    "port": "8080",
                    "path": [
                        "albums"
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Update Album",
            "request": {
                "method": "PUT",
                "header": [],
                "body": {
                    "mode": "raw",
                    "raw": "{\r\n    \"id\":10,\r\n    \"title\":\"dddf\",\r\n    \"artist\":\"88888888\",\r\n    \"price\":888.88888\r\n}",
                    "options": {
                        "raw": {
                            "language": "json"
                        }
                    }
                },
                "url": {
                    "raw": "localhost:8080/albums",
                    "host": [
                        "localhost"
                    ],
                    "port": "8080",
                    "path": [
                        "albums"
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Delete Album by ID",
            "request": {
                "method": "DELETE",
                "header": [],
                "url": {
                    "raw": "localhost:8080/albums/10",
                    "host": [
                        "localhost"
                    ],
                    "port": "8080",
                    "path": [
                        "albums",
                        "10"
                    ]
                }
            },
            "response": []
        }
    ]
}

More sharing of learnings to come ...
Happy unlearn.learn,relearn :)
Thank you.

Basics of OAuth2.0 protocol

Ramu Mangalarapu — Sun, 16 Jan 2022 06:24:15 +0000

Hi,

Today, let's try to understand what is OAuth and why use it.

Whenever you login into any website or mobile App, you see the options like Login with Google, Facebook, Twitter right ?

The mechanisms used behind those options are OAuth. Let's take one example and follow it till the end to understand what OAuth really means.

Suppose you want to Sign Up for new App where you can order food online (food ordering App). You visited their website. In OAuth terms we call this client Application. Now, you can see option Login with Google and you clicked on it. Here Google is Authorization Server or/and Resource Server and your the Resource Owner.

When you can click of food ordering website (OAuth client Application) will redirect you to the Google Authorization Server login page.

Before this our food ordering Application registers with the Google by asking could you please provide authentication to my Application, then Google says Oka, Sure and gives you the Client ID (App ID) and Client Secret (App Secret). These two things should be used by our client App whenever there is authentication involved with Google. Client App should never expose it's credentials in front end.

Now, you as Resource Owner (because you have resources on Authorization Server) Sign In into Google, then Google prompt you saying, Do you want to allow this food ordering App to access your profile picture, email and language settings like some data requested by food ordering App (the things requested by the food ordering App for your data is called Scopes, such as email, phone, etc. ). Google will also display the link to Privacy policy of our client App (food ordering App) and terms of services.

Usually you proceed to give consent to the scopes requested by client Application.

Example of consent screen from Google.

Now, our Client App i.e. food ordering App have access to yours profile image, email and language settings.

Then our food ordering App redirects you to the listings of restaurants near by you to order.

Sounds simple right ?

Yes it is simple if we can understand the roots of it and core problem it is solving. In fact we call the above process called access delegation. How ? because you as resource owner giving access to your data to the client application from your authorization server in return for what ?.

Let's dig bit deeper into it !

Official website of OAuth: https://oauth.net/

And finally this link without fail :)
https://developer.okta.com/blog/2017/06/21/what-the-heck-is-oauth

Now simply, we could say OAuth is authorization framework (protocol).

Current Version of OAuth is OAuth2.0

You can understand the history about OAuth here

Let's try to understand OAuth2 terms involved in it. As it's framework, we need to follow the documentation such rfc.
Here are the few terms you need to understand without fail :

Resource Owner: You or actual user who wants to access Apps where they provide Login with X options (USER).

Client App: Website or mobile Application who wants to enable Login with X option on their login pages (eg, website a-non-existing-food-ordering.com).

Authorization Server: The one who has Resource Owner data such as Google, Facebook, Twitter etc... (gooogle.com)

Resource Server: The one which provides data to the client Apps based on the token they receive, sometimes both Authorization Server and Resource Server can be same (google.com).

Now, let's understand the URL flows involved in it and try to see different terms in it. Please follow below excellent OAuth2.0 Playground and you can read complete website, it is really good resource and sponsored by Okta.

Please visit here without fail to understand the all different OAuth2.0 flows and you can play with them.

https://www.oauth.com/playground/

Hope you got some understanding about OAuth protocol now.

Thank you for reading it.
I wish all the best for your goals.
Happy Learning and Coding :)

OpenID Connect with Okta in Golang

Ramu Mangalarapu — Fri, 14 Jan 2022 15:51:31 +0000

Hello,

If you do not have any idea about Authentication, Authorization, OAuth and OpenID Connect, please see my previous article about basic of IAM.

In this article, I am going to discuss simple steps how we can let the users authenticate with Okta and get the claims.

Note: This article strictly intended to the members who have OAuth, OpenID Connect and Golang knowledge.

Let's say your building application for company X whose data store and all IAM solutions are in Okta platform and now the new application your building for them should authenticate with Okta.

In such cases, we can do OAuth and fetch the tokens corresponding to the user.

We can use those tokens (claims, usually bearer access token and id token) and decode them to as per use case

Follow this to understand how we use Okta APIs for authentication.

Read OAuth2 Simplified for better understanding.

Here we are doing OpenID Connect to fetch user details and this means we are already authenticated the user with Okta.

Sounds confusing between OIDC and OAuth, yes, it is confusing but main difference you need to remembers is following things

1) OIDC is on top of OAuth2
2) OIDC can give you userinfo details
3) OIDC tells that user is authenticated

See here from Okta without fail: https://developer.okta.com/blog/2019/10/21/illustrated-guide-to-oauth-and-oidc

Let's write some code to explain OIDC.

Let's initialize Golang project by

go mod init example.com/okta-golang-oauth2-demo

This will create go.mod and sub sequent adding of packages will also create go.sum.

and create main.go, .env and oktaAPIs.go

Let's create required .env variables like below.

ISSUER_URL=YOUR_OKTA_DEV_DOMAIN/oauth2/default
CLIENT_ID=YOUR_OKTA_CLIENT_ID
CLIENT_SECRET=YOUR_OKTA_CLIENT_SECRET
REDIRECT_URI=http://localhost:9000/authorization/callback
SCOPE=openid profile email
NONCE=somenonce
STATE=somestate
RESPONSE_TYPE=code
AUTHORIZATION_CODE=authorization_code

main.go


package main

import (
    "fmt"
    "log"
    "net/http"
    "net/url"
    "os"
    "text/template"

    "github.com/joho/godotenv"
)

var (
    // kept global because multiple functions can access them
    Issuer, ClientID, ClientSecret, RedirectURI, Scope, Nonce, State, ResponseType, AuthorizationCode string
)

func main() {
    //
    // Usually OAuth2.0 flow starts like below
    // First User (Resource Owner) tries to access client Application.
    // But Client Application checks the session if user is already logged in or not
    // If logged in, it will redirect the user request to page
    // If not user has to sign up using provided Sign In with options like Google, Facebook, Twitter etc..
    // Once your clicks on the "Login with X (Google)", it redirect the user to Authorization Server (here Google)
    // User enters creds and AS validates them, and Google asks the user, do you want this App
    // To read your profile, see your profile image, based on client requested scopes
    // And Google provides link to privacy policy and data usuage of the client App.
    // If resource owner (user) agrees to it, Google redirects to client redirection url.
    // Client takes the code and state from Google and verifies them, it exchanges them
    // fot tokens, Google gives some tokens about the user with all the scopes consented by the user.
    // Now client App can take decisions with the token in its hand of logged in user.

    // Loading env should be done first, because we are going to use
    // use the fields throught the program.
    err := godotenv.Load()
    if err != nil {
        log.Fatal("Error loading .env file")
    }

    Issuer = os.Getenv("ISSUER_URL")
    ClientID = os.Getenv("CLIENT_ID")
    ClientSecret = os.Getenv("CLIENT_SECRET")
    RedirectURI = os.Getenv("REDIRECT_URI")
    Scope = os.Getenv("SCOPE")
    Nonce = os.Getenv("NONCE")
    State = os.Getenv("STATE")
    ResponseType = os.Getenv("RESPONSE_TYPE")
    AuthorizationCode = os.Getenv("AUTHORIZATION_CODE")

    // This is start of OAuth2.0 flow.
    // This is the home handler of our demo where we redirect the user to Okta UI
    http.HandleFunc("/", Index)

    // Now if Okta login is successful, Okta redirects to URL we specified in dashboard
    // we call this URL as Sign In Redirection URL.
    // This is usual OAuth2.0 callback handler process.
    // https://developer.okta.com/docs/reference/api/oidc/#authorize
    // We should check for 'code' and 'state' in query params.
    // And fetch tokens with the code.
    http.HandleFunc("/authorization/callback", Callback)

    log.Println("Starting the Go server at port :9000")
    log.Fatalf("Failed to start the server: %v\n", http.ListenAndServe(":9000", nil))
}

// Index generated Okta Authorization URL.
func Index(rw http.ResponseWriter, r *http.Request) {
    log.Println("Index of the Go Server")
    t, err := template.ParseFiles("index.html")
    if err != nil {
        fmt.Fprint(rw, fmt.Errorf("failed to parse 'index.html': %v", err).Error())
    }

    // This is hardcoded and manual :(.
    // Make sure you append "?" at first and "&" in between query params.
    // Alternatively you can also use http.NewRequest function to build request object
    // and then append query params using url.Values and encode them.
    // See my post: https://dev.to/ramu_mangalarapu/modifications-to-url-in-golang-4m80
    //
    // Now, OAuth2.0 flows start here, for more info, please see below URL.
    // See: https://developer.okta.com/docs/reference/api/oidc/#well-known-oauth-authorization-server
    authorizationURL := Issuer + "/v1/authorize?" + "client_id=" + ClientID + "&redirect_uri=" + RedirectURI + "&scope=" + Scope + "&nonce=" + Nonce + "&state=" + State + "&response_type=" + ResponseType
    log.Printf("Authorization Server URL is: %s", authorizationURL)
    if _, err := url.Parse(authorizationURL); err != nil {
        log.Fatalf("Invalid authorization URL: %s and error: %v", authorizationURL, err)
    }
    t.Execute(rw, authorizationURL)
}

// Callback is normal OAuth2.0 callback handler.
// Usually Authorization Server returns code, state and other params (in HTTP GET)
// to client App.
//
// With the received code and state, client App need to request for token.
func Callback(rw http.ResponseWriter, r *http.Request) {
    log.Println("Okta login is successful, we have got the code and state")
    authorizationServerReturnedState := r.URL.Query().Get("state")
    if authorizationServerReturnedState == "" {
        // So, let us fatal, as we cannot proceed
        log.Fatalf("Received empty 'state' or missing 'state' from Okta Authorization URL")
    }

    // let's compare if state returned by authorization server is matching the one
    // we sent initially, let's compare them
    if authorizationServerReturnedState != State {
        log.Printf("Expected %s: but got: %s", State, authorizationServerReturnedState)
        log.Fatalf("Received malformed 'state' from authorization server")
    }

    code := r.URL.Query().Get("code")
    if code == "" {
        fmt.Fprint(rw, "Received empty 'code' query param or missing 'code' from Authorization Server")
    }

    claims, err := fetchTokens(r, code)
    if err != nil {
        log.Fatalf("Failed to fetch tokens from Okta: %v", err)
    }

    log.Printf("Received response from Okta Token URL is :\n%v", claims)

    // this is to parse and represent the claims in bare html page.
    // or you could use those token, verify them or pass it other functions to let the flow
    // continue.
    t, err := template.ParseFiles("tokens.html")
    if err != nil {
        fmt.Fprint(rw, fmt.Errorf("failed to parse 'tokens.html': %v", err).Error())
    }

    // A different check but this would be useful if there is any
    // failure in calling API.
    if len(claims) == 0 {
        log.Fatal("Did not receive claims from Token endpoint")
    }

    // Check if key exists or not?
    rawJWT, exists := claims["access_token"]
    if !exists {
        log.Fatal("Could not find the key 'access_token' in Okta Token response")
    }

    // Check if it is of type "string" or not?
    jwt, isString := rawJWT.(string)
    if !isString {
        log.Fatalf("'access_token' expected to be string but found %T", jwt)
    }

    // Verifying the token to ensure it is received from Okta
    _, err = verifyAccessToken(jwt)
    if err != nil {
        log.Fatalf("Received token which is not signed By Okta: %v", err)
    }

    u, err := fetchUserInfo(jwt)
    if err != nil {
        log.Fatalf("Failed to fetch user details from Okta /userinfo URL: %v", err)
    }
    t.Execute(rw, u)
}

oktaAPIs.go

package main

import (
    "encoding/base64"
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "net/url"

    verifier "github.com/okta/okta-jwt-verifier-golang"
)

func fetchTokens(r *http.Request, code string) (map[string]interface{}, error) {
    // Basic Authentication Header
    // See: https://en.wikipedia.org/wiki/Basic_access_authentication
    authHeader := base64.StdEncoding.EncodeToString(
        []byte(ClientID + ":" + ClientSecret))

    // Add required query params
    q := r.URL.Query()
    q.Add("grant_type", AuthorizationCode)
    q.Set("code", code)
    q.Add("redirect_uri", RedirectURI)

    // Token URL of Okta and how we need to call it.
    // Please see:https://developer.okta.com/docs/reference/api/oidc/#token
    url := Issuer + "/v1/token?" + q.Encode()
    req, err := http.NewRequest(http.MethodPost, url, nil)
    if err != nil {
        return nil, fmt.Errorf("could not build new HTTP Request object: %v", err)
    }

    h := req.Header
    h.Add("Authorization", "Basic "+authHeader)
    h.Add("Accept", "application/json")
    h.Add("Content-Type", "application/x-www-form-urlencoded")
    h.Add("Connection", "close")
    h.Add("Content-Length", "0")

    log.Printf("Okta Token URL is: %s", req.URL.String())
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return nil, fmt.Errorf("could not do HTTP POST request to Okta Token URL: %v", err)
    }

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return nil, fmt.Errorf("could not read HTTP Response body from Okta Token URL: %v", err)
    }
    // Alawys close response body
    defer resp.Body.Close()

    // why map, becuase it similarly represents the JSON in Golang :)
    claims := make(map[string]interface{})
    json.Unmarshal(body, &claims)
    return claims, nil
}

// fetchUserInfo is to fetch user details by access token.
// Please see: https://developer.okta.com/docs/reference/api/oidc/#userinfo
func fetchUserInfo(token string) (map[string]string, error) {
    userInfoURL := Issuer + "/v1/userinfo"
    if _, err := url.Parse(userInfoURL); err != nil {
        return nil, fmt.Errorf("invalid Okta User Info URL: %s")
    }

    req, err := http.NewRequest(http.MethodPost, userInfoURL, nil)
    if err != nil {
        return nil, fmt.Errorf("Failed to create request object for Okta /userinfo endpoint: %v", err)
    }

    req.Header.Add("Authorization", "Bearer "+token)
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return nil, fmt.Errorf("Failed to send HTTP POST request to Okta /userinfo endpoint: %v", err)
    }

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return nil, fmt.Errorf("Failed to read response body from Okta /userinfo endpoint: %v", err)
    }
    defer resp.Body.Close()

    userDetails := make(map[string]string)
    json.Unmarshal(body, &userDetails)
    return userDetails, nil
}

func verifyAccessToken(t string) (*verifier.Jwt, error) {
    toValidate := map[string]string{}
    toValidate["nonce"] = Nonce
    toValidate["aud"] = "api://default" // why this right? so, please see: https://github.com/okta/okta-jwt-verifier-golang#usage

    jV := verifier.JwtVerifier{
        Issuer:           Issuer,
        ClaimsToValidate: toValidate,
    }

    result, err := jV.New().VerifyAccessToken(t)
    if err != nil {
        log.Println("Invalid Token")
        return nil, fmt.Errorf("%s", err)
    }

    if result != nil {
        log.Println("Valid token")
        return result, nil
    }
    return nil, fmt.Errorf("token could not be verified: %s", "")
}

Now, you can run

go run .\main.go .\oktaAPIs.go to see the UI of link, click on the link and it will redirect you to Okta UI.

Make sure you assign users to this particular App and login with that user, it will fetch user details about him.

Here one more important thing is, please use your environment variables properly, make sure redirect URI specified in Okta Developer dashboard is same as the one your building it.

I will be sharing more of my knowledge in coming articles.

Thanks for reading.
Happy Coding :)

Understanding about Authentication and Authorization

Ramu Mangalarapu — Fri, 14 Jan 2022 12:39:39 +0000

Hi,

Today, I am going to discuss about basic Authentication, Authorization, OAuth and OpenID Connect flows.

First of all we need to understand at fundamental level what is the difference between Authentication and Authorization.

There are tons of articles to explain what these are. Please find below articles that I find interesting and easy to understand.

https://www.okta.com/identity-101/authentication-vs-authorization/

https://auth0.com/intro-to-iam/authentication-vs-authorization/

Let us consider an application (anything Web App, Mobile App, Native App or Home Devices etc.. ) as system.
So in the system, authentication refers whether you exist with the creds you provide authorization is what you can do in the system.

Authentication can only allows you enter into the system but your not allowed to do whatever you want to do, that's where Authorization comes into picture.

Basically Apps assigns roles to the users of the Application to distinguish different types of users like, Root, super user,
admin, manager etc..

Here is the sample representation in picture format

To achieve Authentication and Authorization, mainly application use following concepts (As per my knowledge).

OAuth
OpenID Connect
SAML
Basic Authentication (user enter user name and password)

Now, you have some basic idea about Authentication and Authorization, let's understand how various Apps use these mechanisms to secure their data (services).

You must have seen websites or mobile Apps where you have option to login with Google or Facebook etc..

This is where OAuth and OpenIDConnect flows comes into picture.

Check OAuth playground: https://www.oauth.com/playground/

Please see here about OAuth2.0 and OpenID Connect understand them.

Beginners guide to SAML

Difference between all of them : https://www.okta.com/identity-101/whats-the-difference-between-oauth-openid-connect-and-saml/

Try to understand how the credentials we enter goes through URLs in developer console of the browser, you can even preserve the Logs of console.

Try to see Request URLs, Request Headers, Request Payload and Response Headers and Response Body we receive in the console.

Authentication you do in any website or App either follows OAuth2 or OpenID Connect or SAML or Basic Authentication.

Now, you have good understanding about IAM.

We can learn many things by understanding about Authentication and Authorization just observing developer console (We could see what kind of platforms the application using, what is the flow they are following).

Now-a-days companies (not all the IAM solutions) who build apps are using SaaS solutions to delegate Authentication and Authorization (Just Google how many companies are there for this services) which makes them easy for companies as they can only concentrate on core business logic.

I will be writing simple steps in next article about how we can authenticate the users with Okta in Golag.

Authentication and Authorization is crucial for any application and it is utmost important such that if not implemented properly, it will incur huge losses to company. Company may loss trust of users, partners and different stakeholders.

Thank you for reading.

I wish you all the best for your goals.
Happy Coding :)

Simple Reading of JSON file in Golang.

Ramu Mangalarapu — Sat, 08 Jan 2022 19:02:57 +0000

Hi,

Today I am going to write simple tutorial about reading file in Golang.

Let's say you want to read json file, you can read using golang os ReadFile.

The json file we are reading is same as in our golang code file.

user.json

{
    "id": "0000001",
    "name": "user",
    "email": "user.email.com",
    "phoneNumber": "+0001"
}

main.go

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "log"
    "os"
)

// User is sample object
type User struct {
    ID          string `json:"id"`
    Name        string `json:"name"`
    Email       string `json:"email"`
    PhoneNumber string `json:"phoneNumber"`
}

func main() {
    // provide the file path to read from
    b, err := os.ReadFile("user.json")
    if err != nil {
        log.Fatalf("Failed to read file: %v\n", err)
    }

    // where to store the file contents
    var u User
    json.NewDecoder(bytes.NewBuffer(b)).Decode(&u)

    // printing the file contents or use it for
        // other purposes.
    fmt.Printf("%#v", u)
}

This way can de-serialize to our native Golang types and use them like passing it other functions or doing some computation on it.

Thank you.

Simple REST API in Golang on Contact (non persistent and no authentication)

Ramu Mangalarapu — Sun, 02 Jan 2022 10:06:07 +0000

Hi,

Today, let us discuss how to build simple non persistent (no database connection) and no authentication required REST API in Golang.

Note: This is for totally beginners who wants to learn how to build REST APIs in Golang.

Building a web application is same in almost all the backend programming languages.

We usually look for web frameworks if we want to build REST APIs, in case of Golang, we have
echo,
gin,
fibre, etc.

we can also build applications directly without using any frameworks (violates the DRY principal).

Usually framework supports all the required components to build application such as routing, famous wire format parsers, middlewares, data base connectors, http clients and etc..

As we know REST APIs are nothing but just collection of resources. Here our resource is Contact, which can have fields like name, phone number, alternate phone number and email.

We are going to perform HTTP Get, Post, Put and Delete on this Contact resource. Here in the API we are not using any web framework but we are using 3rd party router library.


package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net/http"

    "github.com/julienschmidt/httprouter"
)

// Contact hold the contact resource collection.
// It is the basic building of this service.
type Contact struct {
    Name                 string `json:"name"`
    PhoneNumber          string `json:"phone_number"`
    AlternatePhoneNumber string `json:"alternate_phone_number"`
    Email                string `json:"email"`
}

// Contacts holds the list of contacts.
type Contacts []Contact

// Response useful to return unified result to client.
type Response struct {
    Status     string                 `json:"status"`
    StatusCode int                    `json:"status_code"`
    Error      string                 `json:"error"`
    Data       map[string]interface{} `json:"data"`
}

// Note: To make things simple, we are using in memory data base here.
var contacts Contacts = []Contact{
    {"Pankaj", "12345", "6789", "pankaj@example.com"},
    {"Suraj", "12345", "6789", "suraj@example.com"},
    {"Anuj", "12345", "6789", "anuj@example.com"},
    {"Raj", "12345", "6789", "raj@example.com"},
}

// Begineers understanding about client server communication about REST APIs:
//
// Usually developers builds Swagger (can be other thing also) API documentation (it tells about
// what is the URL consists of, payload and expected response, error codes, any auth mechanism enforces, different errors).
//
// Usual flow in building "REST APIs".
// In handler code, we receive (access to) req object along with URL params, query string
// and facility to write to response.
// We extract the required path params, query strings and request body (or we can call request payload).
// We usually receive in data interchange format like JSON, YML or etc..
// We then deserialize to our native Golang data types. We validate, santize the data.
// If the data does not match with our validations, we throw 400 (client error).
// Then based on the business usecase either we store in the database (like all Read, Write, Update and Delete operations)
// or we call another REST APIs, we even do some computation with the data in return confirmed data interchange format.
// More in coming posts.

// Index is the starting endpoint of our contacts API.
func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    log.Println("I am in Index") // happiness of developer is proportional to the number logs in code, so always log things
    writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"key": "Someone doing same thing in another parallel universe"})
}

// GetAllContacts fetches all the contact list.
func GetAllContacts(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    log.Println("I am in GetAllContacts")
    writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": contacts})
}

// GetContact fetchs contact by email.
func GetContact(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    log.Println("I am in GetContact")

    // let's extract the path param
    email := ps.ByName("email")
    found, index := fetchContactIndexByEmail(email)
    if found {
        writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": contacts[index]})
        return
    }
    writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": "no data found with the email address"})
}

// CreateContact creates the new contact.
func CreateContact(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    log.Println("I am in CreateContact")
    c, err := readContact(r)
    if err != nil {
        writeResponse(w, "Failure", http.StatusOK, fmt.Errorf("invalid JSON: %v", err).Error(), nil)
        return
    }
    // let us add to contacts list
    contacts = append(contacts, c)
    writeResponse(w, "Success", http.StatusCreated, "no error", map[string]interface{}{"result": c})
}

// UpdateContact updates the contact if there is already one or it creates new one if
// none exists with the given email address.
func UpdateContact(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    log.Println("I am in UpdateContact")
    c, err := readContact(r)
    if err != nil {
        writeResponse(w, "Failure", http.StatusOK, fmt.Errorf("invalid JSON: %v", err).Error(), nil)
        return
    }

    found, index := fetchContactIndexByEmail(c.Email)
    if !found {
        // we did not find the contact, so, let's create new one
        contacts = append(contacts, c)
        writeResponse(w, "Success", http.StatusCreated, "no error", map[string]interface{}{"result": c})
        return
    }

    // contact we want to update
    contacts[index].Name = c.Name
    contacts[index].Email = c.Email
    contacts[index].PhoneNumber = c.PhoneNumber
    contacts[index].AlternatePhoneNumber = c.AlternatePhoneNumber
    writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": c})
}

// DeleteContact deletes contact by email.
func DeleteContact(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
    log.Println("I am in DeleteContact")

    // let's extract the path param
    email := ps.ByName("email")
    found, index := fetchContactIndexByEmail(email)

    // let's delete the contact by the index
    contacts = removeContactByIndex(contacts, index)
    if found {
        writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": "Successfully deleted the contact"})
        return
    }
    writeResponse(w, "Success", http.StatusOK, "no error", map[string]interface{}{"result": "no data found with the email address"})
}

// writeResponse is common utility to write response back to the clients.
func writeResponse(res http.ResponseWriter, status string, statusCode int, err string, data map[string]interface{}) error {
    res.Header().Set("Content-Type", "application/json")
    return json.NewEncoder(res).Encode(Response{
        Status:     status,
        StatusCode: statusCode,
        Error:      err,
        Data:       data,
    })
}

// removeContactByIndex removes the contact and updates the contact list.
func removeContactByIndex(cs Contacts, index int) Contacts {
    return append(cs[:index], cs[index+1:]...)
}

// fetchContactIndexByEmail finds the contact index in list.
func fetchContactIndexByEmail(email string) (found bool, index int) {
    for i := 0; i < len(contacts); i++ {
        if contacts[i].Email == email {
            found = true
            index = i
        }
    }
    return
}

// readContact is a utility for creating and updating contact.
func readContact(r *http.Request) (Contact, error) {
    var c Contact
    if err := json.NewDecoder(r.Body).Decode(&c); err != nil {
        return Contact{}, err
    }
    return c, nil
}

func main() {
    // like all the backend programming languages, Go also has router
    // her we are using 3rd party router.
    router := httprouter.New()

    // supported endpoints
    // names are self explanatory
    router.GET("/", Index)
    router.GET("/contacts", GetAllContacts)
    router.GET("/contacts/:email", GetContact)
    router.POST("/contacts", CreateContact)
    router.PUT("/contacts", UpdateContact)
    router.DELETE("/contacts/:email", DeleteContact)

    // let's start the server and wrap in Fatal routine.
    log.Println("Starting the server at :8080 port")
    log.Fatal(http.ListenAndServe(":8080", router))
}

Total code link as Gist:
https://gist.github.com/developer1622/80820a946bd7986a5464e52eb098e7d5

In case if we want to test this APIs, please import the collection from this link and play around with it in postman client:

https://gist.github.com/developer1622/025eae8ade628ffc8a5d5238371958ad

I wish you all the best in your life.

I will keep sharing my knowledge about Golang in coming articles.
Thank you.

Modifications to url in Golang

Ramu Mangalarapu — Sun, 26 Dec 2021 08:42:46 +0000

Hi,

Today, I am going to write simple steps to modify url in Golang.

Sometimes in Golang server code, we need to redirect to different endpoint where we are required append few query params or request headers before we do redirect.

Please ignore if there are any mistakes.

package main

import (
    "fmt"
    "log"
    "net/http"
    "net/url"
)

func main() {
    // This is just some example URL we received in our backend code.
    // Now we need to append some query params to it,
    // so that we can redirect or hit the newly formed URL.
    u := "https://www.example.com/path1/path2"

    // Scenario (1):
    // We just need to append path. For that we can just append "path" with simple string.
    // Before you add "/" check the URL has already or not
    pAppendedURL := u + "/pathN"
    if isValid, err := IsUrl(pAppendedURL); !isValid {
        log.Fatalf("Invalid URL: %v\n", err)
    }
    fmt.Printf("Appended path param to the URL: %s\n", pAppendedURL)

    // Scenario (2):
    // We need to append one or more query params.
    queryParms := url.Values{}
    queryParms.Add("key1", "value1")
    queryParms.Add("key2", "value2")

    // Append '?' if it does not exists already.
    qAppendedURL := u + "?" + queryParms.Encode()
    if isValid, err := IsUrl(qAppendedURL); !isValid {
        log.Fatalf("Invalid URL: %v\n", err)
    }
    fmt.Printf("Added query params to the URL: %s\n", qAppendedURL)

    // You may want to build request object like below
    // and redirect to the URL you built.
    req, err := http.NewRequest(http.MethodGet, qAppendedURL, nil)
    if err != nil {
        log.Fatalf("Failed to create request object: %v\n", err)
    }

    // Add request headers
    req.Header.Add("header1", "value1")
    req.Header.Add("header2", "value2")

    // So, you can redirect to the URL by new request object
    http.Redirect("your response writer goes here", req, qAppendedURL, 304)
}

// IsUrl checks if the URL is valid or not.
// https://stackoverflow.com/questions/25747580/ensure-a-uri-is-valid/25747925#25747925.
func IsUrl(str string) (bool, error) {
    u, err := url.Parse(str)
    return err == nil && u.Scheme != "" && u.Host != "", err
}

Thank you

http.Post Golang example

Ramu Mangalarapu — Sat, 25 Dec 2021 16:12:28 +0000

Hello,

Today, I am going to write small tutorial about basic usage of http.Post method in Golang. Please ignore if there is any mistakes, thank you.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
)

// Post holds the post data that we send or receive from the API server.
type Post struct {
    ID     string `json:"id,omitempty"`
    Title  string `json:"title"`
    Body   string `json:"Body"`
    UserID string `json:"user_id"`
}

func main() {
    // Before we create resource or post on the API server, we need to see the documentation of the REST API we want to use.
    // Like what is the URL, version, what are the required fields in the resource object.
    // Most of the companies create API docs,  it will have examples in different programming languages along with cURL commands.
    // Companies also create SDKs in different programming languages.
    // Difference between SDK and API: https://addevice.medium.com/difference-between-an-api-and-an-sdk-anyone-can-understand-3a95bf7fc691
    // For example Okta to deal with mangement of users, they are having SDK at:https://github.com/okta/okta-sdk-golang

    // Let's build response body for the server we want to post or create resource
    p := new(Post)
    p.Title = "Title"
    p.Body = "This is body"
    p.UserID = "userID"

    // Now we need erializes Post 'p' to JSON
    b, err := json.Marshal(p)
    if err != nil {
        log.Fatalf("Failed to Serialize to JSON from native Go struct type: %v", err)
    }

    // here http.Post method expects body as 'io.Redear' which should implement Read() method.
    // So, bytes package will take care of that.
    body := bytes.NewBuffer(b)

    // REST APIs means, always think in terms of 'resources'.
    // This is good resource: https://www.practical-go-lessons.com/chap-35-build-an-http-client
    // This post method uses default http client, so timeout is NOT mentioned.
    // Doc of this POST API can be found here: https://jsonplaceholder.typicode.com/guide/.
    postURL := "https://jsonplaceholder.typicode.com/posts"
    resp, err := http.Post(postURL, "application/json; charset=utf-8", body)
    if err != nil {
        log.Fatalf("Failed to create resource at: %s and the error is: %v\n", postURL, err)
    }

    // Always close the response body
    defer resp.Body.Close()

    // Let us just print the response headers info from the server
    log.Printf("Status received from server is: %s", resp.Status)
    log.Printf("StatusCode received from server is: %d", resp.StatusCode)
    log.Printf("Content Type received from Server is: %s", resp.Header["Content-Type"][0])

    // Like I mentioned in my last article,
    // we can read the server response to our native Golang type
    // as the map data structure is close to JSON, we could use it
    // in fact we could use this for most of the wire formats.
    data := make(map[string]interface{})
    if err := json.NewDecoder(resp.Body).Decode(&data); err != nil {
        log.Fatalf("Failed to read response body: %v", err)
    }

    // Let's print the map data by iterating over it.
    // Usually in real usecases we use this response to pass to ther functions.
    for key, value := range data {
        fmt.Printf("%s: %v\n", key, value)
    }
    log.Println("We have successfully created resource and read the response from API server.")
}

Thank you.

Consuming 3rd Party REST APIs Golang {GET method}

Ramu Mangalarapu — Tue, 21 Dec 2021 16:53:42 +0000

Hello World!,

This is my first internet post. So there may be mistakes, please ignore it, thank you.

Today, I am going to write small and simple tutorial about consuming 3rd party API in Golang.

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

func main() {
    // To consume 3rd party API, we need to build request object.
    // This can be anything you want to consume, just paste the URL,
    // usually, we test the URL in API clients such as Postman or through cURL, then
    // we can proceed for consumption.
    req, err := http.NewRequest(http.MethodGet, "https://jsonplaceholder.typicode.com/todos/2", nil)
    if err != nil {
        log.Fatalf("Failed to create request object for /GET endpoint: %v", err)
    }

    // Now add all the required request headers, mostly APIs secured with access tokens
    // req.Header.Add("Authorization", "Bearer "+token)
    // And today's application development, most of wire format is JSON.
    // if the wire format we wanted to be different, we could request but server has to support it.
    req.Header.Add("Content-type", "application/json; charset=utf-8")

    // we can build our custom http clients but here we are using deafult http client
    // when we use custom http client, we can send context object, timeouts etc..
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        log.Fatalf("Failed to send HTTP request: %v", err)
    }

    // here we are reading the response body from the server
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatalf("Failed to read response body: %v", err)
    }

    // Always close the response body
    defer resp.Body.Close()

    // now we need to read this to our native Golang type
    // as the map data structure is close to JSON, we could use it
    // in fact we could this for most of the wire formats.
    data := make(map[string]interface{})

    // this step de-serializes JSON data to our native Golang data
    json.Unmarshal(body, &data)

    // let's print the map data by iterating over it
    for key, value := range data {
        fmt.Printf("%s: %v\n", key, value)
    }
}

Thank you.