DEV Community: Sudipto Biswas

How to Create and Publish a Go Package (Golang)

Sudipto Biswas — Thu, 16 Feb 2023 10:01:42 +0000

1.Create a new project folder

At the command line create and/or browse to a folder where you keep your projects:

Add a new project folder

2.Initialize a Go project
go mod init github.com/dipto0079/rest-API-to-upload-images-Golang

3.Create the library files
You need to create some empty files to update in later steps:
README.md ,LICENSE,init.go,.....

Notice that the test file ends with underscore (_) test (_test.go).

This is how the default test tool knows that the file contains tests.

4.Code Description
The code above includes an init function. The init function in Go is a special (and optional) function that is called when the package that it is in is imported. In this case I'm just using it to log a message. But you could use it for things like initializing a random seed, etc.
**
5.Fill in the license file**
Without the license file pkg.go.dev won't display your documentation.

I use the MIT License. You can copy this text and adjust the copyright info line for yourself:

MIT License

Copyright 2022 Sudipto Biswas

Permission is hereby granted, free of charge, to any person obtaining a copy 
of this software and associated documentation files (the "Software"), to deal 
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 
copies of the Software, and to permit persons to whom the Software is 
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in 
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 
THE SOFTWARE.

5 Tips for Optimizing Your Code for Better Performance in Golang

Sudipto Biswas — Mon, 02 Jan 2023 06:30:34 +0000

As a software engineer, it’s important to ensure that your code is efficient and performs well. In Golang, there are several best practices and techniques you can use to optimize your code for better performance. Here are five tips to help you get started:

Use pointers wisely: Golang uses pointers to reference memory locations. While pointers can be useful in certain situations, they can also lead to poor performance if used excessively or incorrectly. For example, using pointers to pass large structs or slices to functions can result in unnecessary memory allocation and copying. Instead, consider passing these types by value or using a reference type such as a slice header.

// Bad: Passing a large slice by pointer
func slowFunction(s *[]int) {
// do something with s
}

// Good: Passing a large slice by value
func fastFunction(s []int) {
// do something with s
}

Avoid unnecessary allocations: Allocating memory is an expensive operation, especially in a high-concurrency environment. To improve performance, try to minimize the number of allocations your code makes. One way to do this is by reusing slices and other reference types instead of creating new ones. // Bad: Allocating a new slice for each iteration for i := 0; i < 100; i++ { s := make([]int, 10) // do something with s }

// Good: Reusing the same slice
s := make([]int, 10)
for i := 0; i < 100; i++ {
// do something with s
}

Use the right data structures: Choosing the right data structure for the task can significantly impact performance. For example, using a map instead of a slice or vice versa can make a big difference in terms of lookup time and memory usage.

// Bad: Using a slice to store key-value pairs
type Pair struct {
key string
value int
}
pairs := []Pair{}

// Good: Using a map to store key-value pairs
pairs := make(map[string]int)

Use concurrency effectively: Golang’s concurrency model is powerful and efficient, but it’s important to use it wisely. For example, using too many goroutines or not properly synchronizing access to shared resources can lead to poor performance.

// Bad: Spawning a new goroutine for each iteration
for i := 0; i < 100; i++ {
go func() {
// do something concurrently
}()
}

// Good: Using a fixed number of goroutines
const numWorkers = 10
jobs := make(chan int, 100)
for i := 0; i < numWorkers; i++ {
go func() {
for j := range jobs {
// do something concurrently
}
}()
}

Profile and benchmark your code: Finally, it’s important to measure the performance of your code to ensure it meets the desired requirements. Golang provides tools such as pprof and go test -bench to help you profile and benchmark your code. These tools will allow you to identify bottlenecks and optimize your code accordingly.

// Using the pprof tool to profile CPU usage
go test -run=^$ -bench=. -cpuprofile=cpu.out
go tool pprof cpu.out

// Using the go test -bench flag to benchmark performance
func BenchmarkMyFunction(b *testing.B) {
for i := 0; i < b.N; i++ {
MyFunction()
}
}
go test -bench=BenchmarkMyFunction

By following these tips, you can improve the performance of your Golang code and ensure it runs smoothly and efficiently. Remember to always consider the specific needs and requirements of your application when optimizing your code.

I hope this blog post helps give you some ideas for optimizing your code for better performance in Golang. Let me know if you have any questions or need further guidance!

TCP Client in Golang

Sudipto Biswas — Fri, 30 Dec 2022 09:09:33 +0000

This article is an extension of previous article in which we saw how to create a TCP server in golang which can communicate with concurrent users. In this article we are going to see how to make a TCP client in golang that sends the request to the server and consumes the response received back, it is similar to client and server terminal chat where until the client doesn’t quit the communication is taking place. We will be using the net package from the standard library, which is a set of core packages that enhance and extend the language. We don’t need to build our own package or download any from a third party.

For using the net package use the import statement to import the package.

import "net"

We will define some variables which will be required to make a tcp connection, and as the variables are not going to change in the future so we will declare them as constants.

const ( HOST = "localhost" PORT = "8080" TYPE = "tcp" )

The above declared constants are HOST which will be localhost in this case, the PORT number where we can access the server here it is 8080 and the TYPE of network that will be used here it will be tcp.

We will create a util function named checkErr where we pass the error and a error message string as an argument and it handles the error.

func checkErr(err error, msg string) { if err != nil { fmt.Println("error: ", msg, " ", err) os.Exit(1) } }

For connecting with the desired server we use the Dial function from the net package which takes two arguments the first being the network type and the second being the address. Dial function returns net.Conn and the error, we handle the error in the separate checkErr function. We use the defer keyword to close the connection when the program is terminated.

conn, err := net.Dial(TYPE, HOST+":"+PORT) defer conn.Close() checkErr(err, "Unable to connect")

After connecting with the server we read the response from the server using the Read function which takes a slice of byte as argument where it stores the data.

buf := make([]byte, 1024) _, err = conn.Read(buf) checkErr(err, "Error reading data") fmt.Println(string(buf))

For sending messages to the server we first take the input message from the terminal using the functions from the bufio package. We first create a Reader and then read the data from the terminal using the ReadString function.

msgReader := bufio.NewReader(os.Stdin) fmt.Print("Enter your text: ") msg, _ := msgReader.ReadString('\n')

After getting the message we use the Write function to send the message to the server.

conn.Write([]byte(msg))

Here is the full code for the same.

`
package main

import (
"bufio"
"fmt"
"net"
"os"
)

const (
HOST = "localhost"
PORT = "8080"
TYPE = "tcp"
)

func main() {
fmt.Println("!! TCP Client in golang !!")

conn, err := net.Dial(TYPE, HOST+":"+PORT)
defer conn.Close()
checkErr(err, "Unable to connect")

buf := make([]byte, 1024)
_, err = conn.Read(buf)
checkErr(err, "Error reading data")
fmt.Println(string(buf))

for {
msgReader := bufio.NewReader(os.Stdin)
fmt.Print("Enter your text: ")
msg, _ := msgReader.ReadString('\n')
conn.Write([]byte(msg))
}
}

func checkErr(err error, msg string) {
if err != nil {
fmt.Println("error: ", msg, " ", err)
os.Exit(1)
}
}
`

Rest API to upload images Golang

Sudipto Biswas — Fri, 02 Dec 2022 10:58:13 +0000

In this article we are going to learn about how we can create a rest api to upload image, we’ll also going to make sure that our api will handle multiple image upload as well and we are also going to set the file upload limit so it won’t make our system unstable, because long file upload can cause the system failure issue so it’s better to add a limit for the file, and we are going to make sure that we only accept image so we also need to add file type checking as well for our api, if you want images and docs from the same api you can remove the type checking filter and api is ready to handle multiple type files.

Steps we need to follow to make our rest api to upload images.

Create a Repository.
Setup go.mod.
Create main.go file
Setup http server using gorilla mux package.
Create the api.
Connect the rest api with http handler.
Test the api.

Let’s follow the steps one by one to create the successful rest api to upload images.

Create a Repository for our rest api

First we need to create the folder where we can set up our codebase.

Create a folder called upload-images-rest-api. Below is the command to create a directory/folder.

$ go mod init upload-images-rest-api

If you are new to Go then you might need to understand the go packaging first to understand what this command does.

Create main.go file

Inside your upload-images-rest-api folder create another file called main.go.

$ touch main.go

as now we have also created our main.go file now we can start development of our http server.

Setup http server using gorilla mux package

For creating server we are going to use gorilla mux package so first we need to download the package first then we can start the development.

Below is the command to download the gorilla mux package.

$ go get -u github.com/gorilla/mux

After the installation of gorilla mux we can start the development of our code.

Create main function.

First we need to write the boiler plate code of Go, below is the code.

package main

func main() {

}

As of now it’s a boiler plate code that’s why we don’t have any package loaded yet.

Create HTTP Server.

Below is the code of the http server.

package main

import (
"log"
"net/http"

"github.com/gorilla/mux"

)

const PORT = "8080"

func main() {
r := mux.NewRouter()
r.HandleFunc("/ping", nil).Methods("GET")
r.HandleFunc("/upload", nil).Methods("POST")

log.Printf("Server is running on http://localhost:%s", PORT)
log.Println(http.ListenAndServe(":"+PORT, r))

}

First we added the PORT variable in which we want our server to run which is 8080, Inside the main function we have called the mux.NewRouter function of our gorilla mux package and then we have created our routes as mentioned in the code and also attached their methods as well, as you can see we have two API, first /pingis to check if our server is alive or not and second /upload is for our main work to upload images, then we're passing our mux router to http.ListenAndServe function and also passing the port on we want our server to run.

If you try to run the server and go to test any of the api you’ll get the error like this because we just passed the path but for the provided path their is no handler which can read that handle that request.

To solve this issue let’s create our a http handler for our /ping api which will be used as a api heartbeat to check if server is running or not.

Let’s add the handler for the /ping route:

func Ping(w http.ResponseWriter, r *http.Request) {
answer := map[string]interface{}{
"messageType": "S",
"message": "",
"data": "PONG",
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(200)
json.NewEncoder(w).Encode(answer)
}

We have added the Ping function to handle the /ping route and inside the Ping function we have added our response structure as map[string]interface so we can add dynamic response as we want, we're not dependent on struct. We have added messageType, message and data as our response we going to use the same response json for our /upload api expect that data will be going to a struct with multiple fields.

In next we’re write our header content type and http code and then encoding and directly returning back our response as json.

Add Ping function into handler

Now we have our Pingfunction let's add this in our /ping handler.

r.HandleFunc("/ping", Ping).Methods("GET")

Now we have added our Ping handler as well, Let's test the API.
You can test the api using postman or your browser as ping doesn't accepts any parameter so it can be test on browser as well.

/ping is working as expected, now it's time to implement our main handler as well.

Create the api to upload images

Below is the full code of upload image rest api

// handler to handle the image upload
func UploadImages(w http.ResponseWriter, r *http.Request) {
// 32 MB is the default used by FormFile() function
if err := r.ParseMultipartForm(BULK_FILE_SIZE); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}

// Get a reference to the fileHeaders.
// They are accessible only after ParseMultipartForm is called
files := r.MultipartForm.File["file"]

var errNew string
var http_status int

for _, fileHeader := range files {
    // Open the file
    file, err := fileHeader.Open()
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusInternalServerError
        break
    }

    defer file.Close()

    buff := make([]byte, 512)
    _, err = file.Read(buff)
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusInternalServerError
        break
    }

    // checking the content type
    // so we don't allow files other than images
    filetype := http.DetectContentType(buff)
    if filetype != "image/jpeg" && filetype != "image/png" && filetype != "image/jpg" {
        errNew = "The provided file format is not allowed. Please upload a JPEG,JPG or PNG image"
        http_status = http.StatusBadRequest
        break
    }

    _, err = file.Seek(0, io.SeekStart)
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusInternalServerError
        break
    }

    err = os.MkdirAll("./uploads", os.ModePerm)
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusInternalServerError
        break
    }

    f, err := os.Create(fmt.Sprintf("./uploads/%d%s", time.Now().UnixNano(), filepath.Ext(fileHeader.Filename)))
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusBadRequest
        break
    }

    defer f.Close()

    _, err = io.Copy(f, file)
    if err != nil {
        errNew = err.Error()
        http_status = http.StatusBadRequest
        break
    }
}
message := "file uploaded successfully"
messageType := "S"

if errNew != "" {
    message = errNew
    messageType = "E"
}

if http_status == 0 {
    http_status = http.StatusOK
}

resp := map[string]interface{}{
    "messageType": messageType,
    "message":     message,
}
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http_status)
json.NewEncoder(w).Encode(resp)

}

We have added our new handler UploadImages and added the check for limited size data upload for our /upload endpoint, we're passing the BULK_FILE_SIZE into r.ParseMultipartForm function.

In next step we’re getting the all uploaded file using r.MultipartForm.File["file"] and it's giving us the map[string][]*multipart.FileHeader on which we're iterating our loop sequentially.

Inside the loop first we’re opening the file using fileHeader.Open() and processing the returned information file, next we're reading the opened file information in chunks buff := make([]byte, 512) using file.Read(buff) function: we're passing our opened file to Read method and also passing the bytes we need to read from the opened file.

After reading small chunk from file we’re passing that chunks http.DetectContentType function and it's returning back the file type and in next step we're checking file type, we're only accepting JPEG, JPG and PNG images.

In next step we’re calling file.Seek(0, io.SeekStart) for seeking image data fron given offset to whence, then we're creating the uploads folder in the root level of the project, after creating folder we're creating the file where we can save the image data we have opened, and in next we're calling io.Copy(f, file) and passing the data file into our newly created file f.

In the end of the function we’re just processing the request and we need if the function got any error processing the image then it’ll return the error otherwise it’ll return back the successfull message as type json response.

Connect the rest api with http handler

Now we have our handler for /upload api but it's not connected yet so let's connect it and then we'll test our code, We just need to add the UploadImages function as second argument into our /upload handlefunc.

r.HandleFunc("/upload", UploadImages).Methods("POST")

Test the API.

github repositories