DEV Community: Sebastian Roy

Sending an Image as POST Request with Swift 5 to Go 1.19 Server

Sebastian Roy — Wed, 26 Oct 2022 15:33:07 +0000

In this post we will upload an image from the user of an iOS app to a server written in Go as an asynchronous multiform POST request. This could be for uploading a profile image or detecting objects.

We will just return file size, image width and height in pixels as a basic JSON object. If you want to learn how to cast more complex JSON objects into structs, checkout Relaying a Microservice JSON Response to the Client by Unmarshalling Go Structs.

First, we need to obtain the image we intend to send. This could be done by taking a photo using the camera, selecting an image from the library, or loading an image from a URL. Loading an image from a URL is the easiest way. Then we need to create the POST request, send it and catch any error if something went wrong.

Backend in Go 1.19
Returning a JSON with file name, size, width and height
Frontend in Swift 5
Basic Concepts
Code Skeleton
Selecting an Image
Creating and Sending the HTTP Request
Conclusion
Links

If we can’t accept POST requests that contain an image, any request will remain unanswered. So let’s create a server to do that.

Backend in Go 1.19

First initialise a new Go environment and install the echo framework.

$ mkdir ImageHandlerTutorial && cd ImageHandlerTutorial 
$ go mod init ImageHandlerTutorial 
$ go get github.com/labstack/echo/v4 
$ go get github.com/labstack/echo/v4/middleware

Then create a ImageHandlerTutorial.go file with a basic skeleton for using echo and other imports for writing files or encoding JSON.

package main

import (
    "bytes"
    // Formating strings
    "fmt"

    "io"
    // Multipart POST request
    "mime/multipart"
    // Opening and closing files
    "os"
    "time"

    "image"
    // Register possible image formats
    _ "image/png"
    _ "image/jpeg"

    "encoding/json"
    "io/ioutil"
    "log"
    "net/http"

    // Echo framework
    "github.com/labstack/echo/v4"
    "github.com/labstack/echo/v4/middleware"
)

func main() {
    e := echo.New()

    // Avoid CORS error when testing with redocly
    e.Use(middleware.CORS())

    // API Versioning
    //...

    // POST request handling
    //...

    e.Logger.Fatal(e.Start(":1324"))
}

To provide some structure for the URLs, we create two groups, such that we can create resources under 127.0.0.1:1324/api/v1. The IP will change if it is running on a real server instead of the same local machine the iOS Simulator will run on later.

    // API Versioning
    apiGroup := e.Group("/api")
    apiv1Group := apiGroup.Group("/v1")

Under this group, we now create an endpoint imageanalysis. It will handle the POST request 127.0.0.1:1324/api/v1/imageanalysis.

This also includes a function to handle the image file. We open the data in field file into the source src and write the file to dst on disk.

The way it is written here, different uploads will overwrite the same file name file. This is a bit sketchy, but also doesn’t use much space. Think about deleting the file after analysis, renaming it if you need it later, and what happens when multiple users call the function at the same time.

    apiv1Group.POST("/imageanalysis", func(c echo.Context) error {
        // TODO: Catch error when no file is send in POST Request
        // Source
        file, err := c.FormFile("file")
        if err != nil {
            return err
        }
        src, err := file.Open()
        if err != nil {
            return err
        }
        defer src.Close()

        // Destination
        dst, err := os.Create(file.Filename)
        if err != nil {
            return err
        }
        defer dst.Close()

        // Copy
        if _, err = io.Copy(dst, src); err != nil {
            return err
        }
        // TODO: Why do i need to open the file here again, instead of just passing the reference from os.Create()?
        // TODO: Maybe don't create the file on disk at all, just send it from memory
        buf, err := os.Open(dst.Name())
        if err != nil {
            log.Fatal(err)
        }
        defer buf.Close()

        return ImageRequestHandler(c, buf)
    })

To finish, we start the echo server (but that part is already in the skeleton above)

e.Logger.Fatal(e.Start(":1324"))

Save the file and to run the server open a terminal and execute.

$ go run ImageHandlerTutorial.go

This should output

   ____    __
  / __/___/ /  ___
 / _// __/ _ \/ _ \
/___/\__/_//_/\___/ v4.9.0
High performance, minimalist Go web framework
https://echo.labstack.com
____________________________________O/_______
                                    O\
⇨ http server started on [::]:1324

Go Playground doesn’t support anything with networking, so if you want to try it, you need to install Go on your machine. You can see and download the full listing here.

Returning a JSON with file name, size, width and height

So for practice, let’s return a JSON with the file size in kb and image size in pixels. Here we could also send the image for analysis to another server to do object detection. We would need to perform another POST request, this time from Go to an object detection microservice, and I have written about this in Sending Images in POST request as MultiPart Form from Go to Microservice.

// Send an image to microservice for image analysis
func ImageRequestHandler(c echo.Context, img *os.File) error {
    // TODO: Use user image instead
    log.Print(img.Name())

    body := &bytes.Buffer{}
    writer := multipart.NewWriter(body)
    // TODO: Try to change image name here
    fw, err := writer.CreateFormFile("image", "image.jpg")

    _, err = io.Copy(fw, img)
    if err != nil {
        log.Fatal(err)
    }

    writer.Close()

    // Get filesize of image
    stat, err := img.Stat()
    log.Print("File size: ", stat.Size()/1024, " kb")

    // Get width and height in px for the image
    im, _, err := image.DecodeConfig(img)
    if err != nil {
        fmt.Fprintf(os.Stderr, "%s: %v\n", img.Name(), err)
    }
    log.Print(im.Width)

    // Form JSON response that contains name, size, height and width
    jsonStr := fmt.Sprintf(`{ "name": "%s",
                 "size": %d,
                 "width": %d,
                 "height": %d
                }`, img.Name(), stat.Size(), im.Width, im.Height)
    log.Print(jsonStr)

    c.Response().Header().Set("Content-Type", "application/json")
    //TODO: Avoid double casting []byte string?!
    c.Response().Write([]byte(string(jsonStr)))

    // TODO: Figure out how to avoid return and just return with c.Response() similar to w.Write
    return err
}

Basic concepts

Multiform/data http requests

A POST request is one type of HTTP request like GET and PUT. There are many types of POST request, which we can specify in the header. There are image/png or text for example. For flexibility, however, we will use multiformat/data. This offers multiple data fields.

HTTP requests have a boundary. It tells the parser that interesting data is coming and when it sees the same boundary again that this is the end of that data batch. We will define it using a random uuidString. This is just a common way to generate random text to uniquely identify something and can look like 633a76d8-87a3-494c-bd1c-7192985e01f5.

Asynchronous tasks with DispatchQueues

Like any type of networking activity, it takes some time to execute and might even fail. We want to avoid blocking the user interface waiting for a response. That’s why multithreading is used. This is a way to make asynchronous networking requests. There are many ways to do this and it is a complex subject. Here we will use DispatchQueues. This is a way to schedule tasks with the first-in-first-out principle (FIFO) and we let iOS handle thread management.

Swift Code Skeleton

We will use two functions loadFrom and uploadImage and wrap those up in a class POSTImage. Then we create an object postRequest of our new type POSTImage and execute.

import UIKit

class POSTImage {
    var image = UIImage()

    // Load an image from URL 
    func loadFromURL(URLAddress: String) {

    }

    // Load an image from Library (TODO)
    func loadImageFromLibrary() {}

    // Sending the image to the server
    func uploadImage(fileName: String, image: UIImage) {

    }
}


let postRequest = POSTImage()
postRequest.loadFromURL(URLAddress: "https://upload.wikimedia.org/wikipedia/commons/b/b9/Panther_chameleon_%28Furcifer_pardalis%29_male_Montagne_d%E2%80%99Ambre.jpg")

Selecting an Image

Load an image from URL

So, now let’s get an image by loading it from a URL with loadFromURL(...) with an URLAddress of type String as argument. We specified the URL in the last line of our skeleton above already. In a real scenario, this could be using a URL from a s3 object storage. We use guard to catch errors when parsing the URL.

 guard let url = URL(string: URLAddress) else {
     return
 }

Next we will load the image. Loading an image takes some time and can fail. We would rather not block the user interface, therefore handling the task over to a separate thread with DispatchQueue.main.async. First, we try to obtain the Data from the url into imageData. Then the imageData will be cast into loadedImage which can be used for display as UIImage.

Now we will upload the image. This is a provisional solution, but it works. The issue is that we loaded the image in a separate thread. We can only call uploadImage, once we already got the image. Therefore, our program has to wait or we need a callback function that is called after the image was loaded from URL. We will do that later, but for now, we just call uploadImage from the sub thread in DispatchQueue.

DispatchQueue.main.async { [weak self] in
    if let imageData = try? Data(contentsOf: url) {
    if let loadedImage = UIImage(data: imageData) {
       self?.image = loadedImage
       // TODO: Remove this and use completion handler instead
       self?.uploadImage(fileName: "file", image: loadedImage)
    }
}

The other issue is that we haven’t implemented uploadImage yet, which we will do next. Here is the full listing for the function selectImageFromURL.

func selectImageFromURL(URLAddress: String) {
     guard let url = URL(string: URLAddress) else {
         return
     }

     DispatchQueue.main.async { [weak self] in
        if let imageData = try? Data(contentsOf: url) {
            if let loadedImage = UIImage(data: imageData) {
               self?.image = loadedImage
             // TODO: Remove this and use completion handler instead
               self?.uploadImage(fileName: "file", image: loadedImage)
    }
}

Create and send POST request

Now we create a URLSession named session. It will be sent to the url specified by URL(...). This must be an endpoint to your server, that will handle HTTP POST requests. I have written a previous post about how to do this also with URLSession and DispatchQueues at least with JSON objects (text, no images) in Communications from Go Backend to SwiftUI Frontend with JSON API.

First, we need to define the server URL and start a URLSession to handle networking.

let url = URL(string: "http://127.0.0.1:1324/api/v1/imageanalysis")

let session = URLSession.shared

Now we configure the request as POST request and making it a multipart/form-data request in the header, where we configure how the boundary looks like.

let boundary = UUID().uuidString
var urlRequest = URLRequest(url: url!)
urlRequest.httpMethod = "POST"
urlRequest.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")

After the header we need to set the actual data. We create data of type Data. Then we append the boundary and add the parameter ”file”. The Content-Type of this parameter is image/png. And finally we have to append the actual file data image.pngData. Finally, we close the requests boundary.

var data = Data()

// Add the image data to the raw http request data
data.append("\r\n--\(boundary)\r\n".data(using: .utf8)!)

let paramName = "file"
data.append("Content-Disposition: form-data; name=\"\(paramName)\"; filename=\"\(fileName)\"\r\n".data(using: .utf8)!)
data.append("Content-Type: image/png\r\n\r\n".data(using: .utf8)!)
data.append(image.pngData()!)

data.append("\r\n--\(boundary)--\r\n".data(using: .utf8)!)

Now that we defined the request, it is time to execute it. We use the session.uploadTask with our created urlRequest and data as arguments. Then we handle the response in the completionHandler. There we convert the JSON response into a Go struct.

session.uploadTask(with: urlRequest, from: data, completionHandler: { responseData, response, error in
                if error == nil {
                    let jsonData = try?JSONSerialization.jsonObject(with: responseData!, options: .allowFragments)
                    if let json = jsonData as? [String: Any] {
                        print(json)
                    }
                }
    }).resume()

Now let’s look at the full listing of the function uploadImage.

func uploadImage(fileName: String, image: UIImage) {
        let url = URL(string: "http://127.0.0.1:1324/api/v1/electrophoresis/imageanalysis")
        let boundary = UUID().uuidString

        let session = URLSession.shared

        var urlRequest = URLRequest(url: url!)
        urlRequest.httpMethod = "POST"

        urlRequest.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")

        var data = Data()

        // Add the image data to the raw http request data
        data.append("\r\n--\(boundary)\r\n".data(using: .utf8)!)
        let paramName = "file"
        data.append("Content-Disposition: form-data; name=\"\(paramName)\"; filename=\"\(fileName)\"\r\n".data(using: .utf8)!)
        data.append("Content-Type: image/png\r\n\r\n".data(using: .utf8)!)
        data.append(image.pngData()!)

        data.append("\r\n--\(boundary)--\r\n".data(using: .utf8)!)

        session.uploadTask(with: urlRequest, from: data, completionHandler: { responseData, response, error in
                if error == nil {
                    let jsonData = try? JSONSerialization.jsonObject(with: responseData!, options: .allowFragments)
                    if let json = jsonData as? [String: Any] {
                        print(json)
                    }
                }
            }).resume()
    }

Finally, we get to see the result of all the hard work so far:

["size": 5451466, "name": file, "height": 1396, "width": 2872]

This is the analysis done by the server, shown on the client. Real world analysis would be more complex and the client would display that data on the UI instead of just printing it to the console.

Bonus: Some curious errors

One issue I had was that data function wasn’t found. This was because I created my own data type that was called data. This has overridden the native function. So give your types new names that are not used in iOS. iOS libraries prefix any new class or struct to avoid it.

Full Listing

import UIKit


class POSTImage {
    var image = UIImage()

    func loadFrom(URLAddress: String) {
        guard let url = URL(string: URLAddress) else {
            return
        }

        DispatchQueue.main.async { [weak self] in
            if let imageData = try? Data(contentsOf: url) {
                if let loadedImage = UIImage(data: imageData) {
                     // Why is self optional here?
                     self?.image = loadedImage
                     self?.uploadImage(fileName: "file", image: loadedImage)
                }
            }
        }
    }

    func uploadImage(fileName: String, image: UIImage) {
        let url = URL(string: "http://127.0.0.1:1324/api/v1/imageanalysis")
        let boundary = UUID().uuidString

        let session = URLSession.shared

        var urlRequest = URLRequest(url: url!)
        urlRequest.httpMethod = "POST"

        urlRequest.setValue("multipart/form-data; boundary=\(boundary)", forHTTPHeaderField: "Content-Type")

        var data = Data()

        // Add the image data to the raw http request data
        data.append("\r\n--\(boundary)\r\n".data(using: .utf8)!)
        let paramName = "file"
        data.append("Content-Disposition: form-data; name=\"\(paramName)\"; filename=\"\(fileName)\"\r\n".data(using: .utf8)!)
        data.append("Content-Type: image/png\r\n\r\n".data(using: .utf8)!)
        data.append(image.pngData()!)

        data.append("\r\n--\(boundary)--\r\n".data(using: .utf8)!)

        session.uploadTask(with: urlRequest, from: data, completionHandler: { responseData, response, error in
                if error == nil {
                    let jsonData = try? JSONSerialization.jsonObject(with: responseData!, options: .allowFragments)
                    if let json = jsonData as? [String: Any] {
                        print(json)
                    }
                }
            }).resume()
    }
}

let postRequest = POSTImage()
postRequest.loadFrom(URLAddress: "https://upload.wikimedia.org/wikipedia/commons/b/b9/Panther_chameleon_%28Furcifer_pardalis%29_male_Montagne_d%E2%80%99Ambre.jpg")

// TODO: wait for myimageview to load
// uploadImage(fileName: "file", image: myImageView.image)

Conclusion

To sum things up, we discussed the server and client side of sending images over HTTP with POST requests, doing simple analysis on the server and returning the result as JSON to the client. I hope I could cover this subject in a useful way with most of the puzzle pieces.

Links

https://orjpap.github.io/swift/http/ios/urlsession/2021/04/26/Multipart-Form-Requests.html
https://stackoverflow.com/questions/29623187/upload-image-with-multipart-form-data-ios-in-swift
https://www.hackingwithswift.com/books/ios-swiftui/sending-and-receiving-orders-over-the-internet

How to Document Code with Swimm.io

Sebastian Roy — Fri, 26 Aug 2022 00:53:37 +0000

Code should be documented to avoid building technical debt. Even if other developers join a team with poorly documented code, they tend to document the code less themselves, worsening the problem. It also avoids a unique individual (the founder, for example) being the king over a part of a codebase (which may be why some people don’t document).

The code itself is kind of a documentation already. While not being a complete documentation, lacking insights and reasons a particular choice was made, it does show a lot. That means writing documentation would have to copy a lot of the ideas that are visible in code. That could be the structure of objects and classes. On the other hand, it is cumbersome to go to a separate site for documentation, simply because it is a pain just to find the right spot in the documentation that refers to the section of code I am interested in. And I haven’t even talked about keeping code and documentation in sync.

Swimm.io offers a nice solution to all of this. It references documentation right within VSCode. In the documentation, you can quote code directly. It is automatically updated if any changes are committed to the repository or it at least tells you if any manual adjustment is needed.

So in the last post I have written about documenting APIs with Insomnia and Redocly, now let’s look at documenting the code itself. The setup we will be using VSCode as an editor, a GitHub and swimm.io account (one private repo free, unlimited repos $29 per user per month).

Connect a Repository

First, we need to create an account and connect it to a GitHub repository.

Create a new Document

Now we have access to the codebase from our documentation, we can create a new document and directly select code as a quote in it.

So create a new document and name the document.

Then press / to open option menu and select Code Snippet.

This will let you choose a file from your code. Use your courser to select the code you want to refer in the documentation and klick Add & Exit.

So let's say we aim to refer to a variable name in the code. We could just do that, but what happens when someone changes the name? Then the documentation and the code is out of sync. To avoid that, we can link variables to the code. When you select a keyword and convert the style into code by clicking </>. It then opens a suggestion from which line in the code you want to link the variable to.

After Linking, you can see green checkmarks next to the keyword

Now that we created our first document, we will have to commit it to the repository by clicking „Create a pull request“ and select „Push to master“ and hit Save.

Now that the documentation is stored in our GitHub repository, next we will add the Swimm extension to VSCode and see how our code now looks from our IDE.

View Documentation from VSCode

Open VSCode. In the extension tab, search for Swimm and install it.

Then we log in to our Swimm account from the Swimm extension tab and make sure to sync the git repository and checkout the main branch (or the one you committed the Swimm documentation to).

This is where the work pays off. You note an additional line above the user struct. If we hover over it, we get the caption we set for our code previously.

Now, if we click on it, we directly jump to the documentation section within VSCode.

This is so cool. If we double-click, we can even edit the documentation and save it on the next commit, all without leaving VSCode.

Keeping Documentation and Code in Sync

Now what happens when we rename the field Email to PrimaryEmail.

When refactoring this quoted section, Swimm will issue a warning.

We will not have to Edit Doc and Reselect the code of interest. Then hit Update Selection and Create Pull Request to push to main. All issues should be resolved again.

And this is basically all we need to know on how to document code with Swimm.io.

Bonus: Playlists

Going back to the beginning of this post, regarding onboarding new developers. Playlist give new developers an ordered way to dig into your codebase.

Conclusion

Although we started by documenting code, we discussed the importance of that in the context of building a team to develop a product. I am myself not up to speed yet, but will spend the next couple of days integrating Swimm.io in my routine. Do you have any war stories come to mind? Would love to read them in the comments.

Relaying a Microservice JSON Response to the Client by Unmarshalling Go Structs

Sebastian Roy — Fri, 05 Aug 2022 17:47:00 +0000

In this post we will transform a JSON response from a microservice by adding metadata like a name, date or a unique identifier and relaying it to the user client. This is part of my journey in building AI SaaS app on iOS for researchers, as an example we will pretend to send data to an AI object identifier for zoo animals.

The past week I have been working on connecting an image analysis microservice to the user frontend. We can send an image via post request and retrieve an array of x and y-coordinates as well as a label for the identified object and a confidence parameter.

{"success": true,
 "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Pinguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
}

In this post I will talk about transforming the JSON response and adding metadata like a name, date or a unique identifier. You could also convert the data, so instead of forwarding x_min and x_max coordinates, we could relay X-Min as coordinate and width instead of x_max. In order to distinguish on the client if an object was detected by an algorithm or a human we add a new field method. This could also be a more complicated object to store the version of the AI algorithm. The relayed response should look like this

{"name": "Zoo Image Analysis",
 "xid": "9m4e2mr0ui3e8a215n4g"
 "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "x": 682,
            "y": 109,
            "width": 77, 
            "height": 25,
            "method": "AI"
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "x": 782,
            "y": 209,
            "width":  -13,
            "height": -65,
            "method": "AI"
        }]
}

To do this we will create a ResponseAdapter struct that mimics the response structure of the AI microservice as well as a PredictionAdapter. Then we create a Response and a Prediction struct that contains the fields and methods we want to relay to our client.

Print unknown JSON data with an empty interface
Convert JSON data into a usable struct to access data fields
Rewrite into testable functions
Implementing custom UnmarshalJSON
Conclusion

When retrieving some JSON data from an API, we cannot call those properties like with data.name. Converting this data into structs takes some unmarshalling, mapping and interfacing. So let’s play with some JSON handling examples to step by step arrive at our goal, which is making data from an API call usable.

You can follow these steps using your local tooling in VS Code and go-tools or just use Go Playground without any installation.

Printing unknown JSON data with an empty interface

To have some sample data, we read some JSON data from an AI microservice as a multiline string into a byte array.

var data = []byte(`{"success": true,
    "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
    }`)

Next we create a temporary empty interface

var temp map[string]interface{}

Then we unmarshall the data into this variable temp. This requires an import of „encoding/json“.

json.Unmarshal(data, &temp)

Catch any unmarshalling errors

if err != nil {
        fmt.Println(err)
}

Then we can print out temp.

fmt.Println(temp)

The result should be

map[predictions:[map[confidence:0.69146144 label:Lion x_max:186 x_min:109 y_max:707 y_min:682] map[confidence:0.7 label:Penguin x_max:196 x_min:209 y_max:717 y_min:782]] success:true]

Great. If we want to access just the title

fmt.Println(temp.predictions[0])

we get an error.

./prog.go:40:19: temp.predictions undefined (type map[string]interface{} has no field or method predictions)

Ok, let’s take care about this next. The listing for this section.

package main

import (
    "encoding/json"
    "fmt"
)

func main() {

    var data = []byte(`{"success": true,
    "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
    }`)

    var temp map[string]interface{}

    //var prediction PredictionsAdapter

    err := json.Unmarshal(data, &temp)

    if err != nil {
        fmt.Println(err)
    }

    //fmt.Println(temp.predictions[0])
}

Accessing data fields

Assuming we know the expected data structure in advance, lets create a struct.

type ResponseAdapter struct {
    Success     bool                 `json:"success"`
    Predictions []PredictionsAdapter `json:"predictions"`
}

Scroll up to the first JSON response to see where this is coming from. This is mirroring the response we get from the AI microservice. To be exported, the names must start with a capital letter. []PredictionsAdapter means that we expect multiple objects or an array of type PredictionsAdapter.

type PredictionsAdapter struct {
    Confidence float64 `json:"confidence"`
    Label      string  `json:"label"`
    YMin       int     `json:"y_min"`
    XMin       int     `json:"x_min"`
    YMax       int     `json:"y_max"`
    XMax       int     `json:"x_max"`
}

We extend the PredictionsAdapter with two functions Width() and Height(), since we need this information later for our client response with Predictions.

func (p PredictionsAdapter) Width() int {

    return p.XMax - p.XMin
}

func (p PredictionsAdapter) Height() int {

    return p.YMax - p.YMin
}

Instead of unmarshalling it into an empty interface, we use a variable of type ResponseAdapter.

var responseAdapter ResponseAdapter

json.Unmarshall(data, &responseAdapter)

fmt.Println(responseAdapter.Predictions[0])

The & implies that we are referring to the address in memory. And thats how we can access JSON data from go as structs :).

{0.69146144 Lion 682 109 707 186}

The full listing

package main

import (
    "encoding/json"
    "fmt"
)

type ResponseAdapter struct {
    Success     bool                 `json:"success"`
    Predictions []PredictionsAdapter `json:"predictions"`
}

type PredictionsAdapter struct {
    Confidence float64 `json:"confidence"`
    Label      string  `json:"label"`
    YMin       int     `json:"y_min"`
    XMin       int     `json:"x_min"`
    YMax       int     `json:"y_max"`
    XMax       int     `json:"x_max"`
}

func (p PredictionsAdapter) Width() int {

    return p.XMax - p.XMin
}

func (p PredictionsAdapter) Height() int {

    return p.YMax - p.YMin
}

func main() {

    var data = []byte(`{"success": true,
    "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
    }`)

    //var temp map[string]interface{}

    var responseAdapter ResponseAdapter

    err := json.Unmarshal(data, &responseAdapter)

    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(responseAdapter.Predictions[0])
}

Testing

I find it always motivating to write something that works and see some result first. To adapt good practices let’s now rewrite everything into a testable function and get rid of main(). If you use Go Playground, it will automatically run the test functions in absence of main().

First import "testing" and "github.com/google/go-cmp/cmp". It is a library to compare expectations want with actual result got of a function. It checks if both are equal with if !cmp.Equal(want, got) and returns an error with the differences otherwise.

The declaration of datawe plug out of the functional context such that it is accessible from each testing function.

Most of what has been happening in mainwe will rename to ParseImageAnalysisResponse which will receive the JSON content of type []byte and return a ResponseAdapter and an error.

The test function has the same name with prefix TestXxx and a reference to *testing.T which enables automated testing in go with go test. The first line then enables parallel test execution with t.Parallel().

Then we declare our expected result. This time we don’t use the JSON format, since we expect data as a go struct. Thats why we state the type before opening curly parenthesises want := ResponseAdapter{... .}

want := ResponseAdapter{
        Success: true,
        Predictions: []PredictionsAdapter{
            {
                Confidence: 0.69146144,
                Label:      "Lion",
                YMin:       682,
                XMin:       109,
                YMax:       707,
                XMax:       186,
            },
            {
                Confidence: 0.7,
                Label:      "Penguin",
                YMin:       782,
                XMin:       209,
                YMax:       717,
                XMax:       196,
            },
        },
}

Then we have to call the function with our data and catch possible errors. got will be of type ResponseAdapter, since this is what ParseImageAnalysisResponseAdapter(…) returns.

got, err := ParseImageAnalysisResponseAdapter(data)
if err != nil {
        t.Fatal(err)
}

And last we compare expectation and result.

if !cmp.Equal(want, got) {
    t.Error(cmp.Diff(want, got))
}

If everything is correct we can use Run on Go Playground or go test in the command line and the test should pass or return an error.

=== RUN   TestParseImageAnalysisResponse
=== PAUSE TestParseImageAnalysisResponse
=== CONT  TestParseImageAnalysisResponse
{0.69146144 Lion 682 109 707 186}
--- PASS: TestParseImageAnalysisResponse (0.00s)
PASS

Full listing

package main

import (
    "encoding/json"
    "fmt"
    "github.com/google/go-cmp/cmp"
    "testing"
)

type ResponseAdapter struct {
    Success     bool                 `json:"success"`
    Predictions []PredictionsAdapter `json:"predictions"`
}

type PredictionsAdapter struct {
    Confidence float64 `json:"confidence"`
    Label      string  `json:"label"`
    YMin       int     `json:"y_min"`
    XMin       int     `json:"x_min"`
    YMax       int     `json:"y_max"`
    XMax       int     `json:"x_max"`
}

var data = []byte(`{"success": true,
    "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
    }`)

func ParseImageAnalysisResponseAdapter(data []byte) (ResponseAdapter, error) {

    var responseAdapter ResponseAdapter

    err := json.Unmarshal(data, &responseAdapter)

    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(responseAdapter.Predictions[0])

    return responseAdapter, nil
}

func TestParseImageAnalysisResponseAdapter(t *testing.T) {
    t.Parallel()

    want := ResponseAdapter{
        Success: true,
        Predictions: []PredictionsAdapter{
            {
                Confidence: 0.69146144,
                Label:      "Lion",
                YMin:       682,
                XMin:       109,
                YMax:       707,
                XMax:       186,
            },
            {
                Confidence: 0.7,
                Label:      "Penguin",
                YMin:       782,
                XMin:       209,
                YMax:       717,
                XMax:       196,
            },
        },
    }

    got, err := ParseImageAnalysisResponseAdapter(data)
    if err != nil {
        t.Fatal(err)
    }

    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }

}

Now we have the data available in such a format that we can actually use it. We could even convert it to JSON and pass it to our user client. But sometimes the implementation logic requires that we add or remove some fields and provide the data in a different format. If that is what makes most sense, we should do it to maintain a well designed data workflow, even if it takes some additional work.

In the next section we will define the types Response and Predictions in a way we actually need them and transform them from our Adapter types using custom UnmarshallJSON. First we will add some custom metadata. Adding or changing fields in []PredictionsAdapter however requires looping through each element in the array.

Implementing custom UnmarshallJSON

Let’s create the Test for ParseImageAnalysisResponse that we are going to implement next (ignore any errors). Out want will be a Response that contains some metadata like name and Xid and the array of predictions. We use the data to pass it to ParseImageAnalysisResponse and than see if our expectation was met.

func TestParseImageAnalysisResponse(t *testing.T) {
    t.Parallel()

    want := Response{
        Name: "Test Image",
        Xid:  "9m4e2mr0ui3e8a215n4g",
        Predictions: []Prediction{
            {
                Confidence: 0.69146144,
                Label:          "Lion",
                X:              109,
                Y:              682,
                Width:          77,
                Height:         25,
                Method:         "AI",
            },
            {
                Confidence:     0.7,
                Label:          "Penguin",
                X:              209,
                Y:              782,
                Width:          -13,
                Height:         -65,
                Method:         "AI",
            },      
        },
    }

    got, err := ParseImageAnalysisResponse(data)
    if err != nil {
        t.Fatal(err)
    }

    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }

}

Next we define types that match the JSON API response we want to relay to our client, that could be a web app or a mobile app for Android or iOS.

type Prediction struct {
    Confidence  float64 `json:"confidence"`
    Label       string  `json:"label"`
    X           int     `json:"x"`
    Y           int     `json:"y"`
    Width       int     `json:"width"`
    Height      int     `json:"height"`
    Method      string  `json:"method"` // AI, Manual, Predicted(?)
}

The Response will contain many of those predictions, since many objects can be detected in an image. But further more it will include meta data like a Name and an Xid. A Xid is a unique identifier that is a related but shorter version of UUID. We might need to change the type later, but for now let’s use string.

type Response struct {
    Name        string       `json:"name"`
    Xid         string       `json:"xid"`
    Predictions []Prediction `json:"predictions"`
}

Now we will adapt the ParseImageAnalysisResponse function. It will still use the data of type []byte as an argument, but instead of a ResponseAdapter it will now return our new Response. Therefore we use a variable response of type Response and use this when unmarshalling. We then check for errors and return the response.

func ParseImageAnalysisResponse(data []byte) (Response, error) {

    var response Response
    // Use custom UnmarshalJSON method to obtain Response
    err := json.Unmarshal(data, &response)

    if err != nil {
        fmt.Println(err)
    }

    return response, nil

}

Since there is no easy 1-to-1 mapping from the ResponseAdapter to the Response, we need to do some manual work. Instead of the default UnmarshalJSON which we don’t have to specify, we will now use a customized version of it that explicitly states how we can derive our Response to the client from the data in the ResponseAdapter.
The data we send to ParseImageAnalysisResponse is the one we want to unmarshall. We know that we can unmarshall it into a ResponseAdapter, so thats what we will do first, but not into the pointer of the response object r, but into a temporary variable tmp.

var tmp ResponseAdapter
err := json.Unmarshal(data, &tmp)

if err != nil {
    fmt.Println(err)
}

This variable also has the Predictions in form of a PredictionsAdapter. We now cast them into a new Predictions-Array.

var p []Prediction

We do this by looping though each element PredictionsAdapter

for _, element := range tmp.Predictions {

and create a temporary corresponding predictions element.
This predictions element also defines the data for any added field or it could also leave out fields from PredictionsAdapter. Then we append the Predictions element to the Predictions Array p, that was defined before the loop. I did variable creation and appending kind of in one step here and closed the loop.

p = append(p, Prediction{
            Confidence: element.Confidence,
            Method:     "AI",
            Label:      element.Label,
            XMin:       element.XMin,
            XMax:       element.XMax,
            YMin:       element.YMin,
            YMax:       element.YMax,
    })
}

Then we set the Response fields. That includes metadata like the name and Xid and setting the predictions array p.

r.Name = "Test Image"
r.Xid = "9m4e2mr0ui3e8a215n4g"
r.Predictions = p

And last since UnmarshalJSON requires us to return an error, we do that as well.

return err

Now lets see how this function looks in its full glory.

func (r *Response) UnmarshalJSON(data []byte) error {


    var tmp ResponseAdapter
    err := json.Unmarshal(data, &tmp)

    if err != nil {
        fmt.Println(err)
    }

    var p []Prediction
    for _, element := range tmp.Predictions {

        p = append(p, Prediction{
            Confidence: element.Confidence,
            Method:     "AI",
            Label:      element.Label,
            X:          element.XMin,
            Y:          element.YMin,
            Width:      element.Width(),
            Height:     element.Height(),
        })

    }

    r.Name = "Test Image"
    r.Xid = "9m4e2mr0ui3e8a215n4g"
    r.Predictions = p

    return err
}

Conclusion

Great. We can now take an image that was annotated with an AI microservice and make this analysis useful for relaying it the user client by enhancing it with more relevant data for a JSON response. On the networking side how to exactly send JSON responses from a HTTP request to a client was explained . I talked about how to send POST requests to the AI micrososervice in a previous post.

Last, let’s look at the full listing, which you can find also on Go Playground. The tests could also be written into a separate file of course.

Thanks for reading, I hope you found this useful. This has been one of the first five posts on my new blog, so I appreciate any comments and suggestions.

package main

import (
    "encoding/json"
    "fmt"
    "github.com/google/go-cmp/cmp"
    "testing"
)

type ResponseAdapter struct {
    Success     bool                 `json:"success"`
    Predictions []PredictionsAdapter `json:"predictions"`
}

type PredictionsAdapter struct {
    Confidence float64 `json:"confidence"`
    Label      string  `json:"label"`
    YMin       int     `json:"y_min"`
    XMin       int     `json:"x_min"`
    YMax       int     `json:"y_max"`
    XMax       int     `json:"x_max"`
}


func (p PredictionsAdapter) Width() int {

    return p.XMax - p.XMin
}

func (p PredictionsAdapter) Height() int {

    return p.YMax - p.YMin
}


type Prediction struct {
    Confidence float64 `json:"confidence"`
    Label      string  `json:"label"`
    X          int     `json:"x"`
    Y          int     `json:"y"`
    Width      int     `json:"width"`
    Height     int     `json:"height"`
    Method     string  `json:"method"` // AI, Manual, Predicted(?)
}


type Response struct {
    Name        string       `json:"name"`
    Xid         string       `json:"xid"`
    Predictions []Prediction `json:"predictions"`
}


func (r *Response) UnmarshalJSON(data []byte) error {


    var tmp ResponseAdapter
    err := json.Unmarshal(data, &tmp)

    if err != nil {
        fmt.Println(err)
    }

    var p []Prediction
    for _, element := range tmp.Predictions {

        // fmt.Println("element", element, "at index", index)

        p = append(p, Prediction{
            Confidence: element.Confidence,
            Method:     "AI",
            Label:      element.Label,
            X:          element.XMin,
            Y:          element.YMin,
            Width:      element.Width(),
            Height:     element.Height(),
        })

    }

    r.Name = "Test Image"
    r.Xid = "9m4e2mr0ui3e8a215n4g"
    r.Predictions = p

    return err
}

var data = []byte(`{"success": true,
    "predictions": [
        {
            "confidence": 0.69146144,
            "label": "Lion",
            "y_min": 682,
            "x_min": 109,
            "y_max": 707,
            "x_max": 186
        },
        {
            "confidence": 0.7,
            "label": "Penguin",
            "y_min": 782,
            "x_min": 209,
            "y_max": 717,
            "x_max": 196
        }]
    }`)

func ParseImageAnalysisResponseAdapter(data []byte) (ResponseAdapter, error) {

    var responseAdapter ResponseAdapter

    err := json.Unmarshal(data, &responseAdapter)

    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(responseAdapter.Predictions[0])

    return responseAdapter, nil
}

func ParseImageAnalysisResponse(data []byte) (Response, error) {

    var response Response
    // Use custom UnmarshalJSON method to obtain Response
    err := json.Unmarshal(data, &response)

    if err != nil {
        fmt.Println(err)
    }
    // fmt.Println("Response")
    // fmt.Println(response)

    return response, nil

}


func TestParseImageAnalysisResponseAdapter(t *testing.T) {
    t.Parallel()

    want := ResponseAdapter{
        Success: true,
        Predictions: []PredictionsAdapter{
            {
                Confidence: 0.69146144,
                Label:      "Lion",
                YMin:       682,
                XMin:       109,
                YMax:       707,
                XMax:       186,
            },
            {
                Confidence: 0.7,
                Label:      "Penguin",
                YMin:       782,
                XMin:       209,
                YMax:       717,
                XMax:       196,
            },
        },
    }

    got, err := ParseImageAnalysisResponseAdapter(data)
    if err != nil {
        t.Fatal(err)
    }

    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }

}

func TestParseImageAnalysisResponse(t *testing.T) {
    t.Parallel()

    want := Response{
        Name: "Test Image",
        Xid:  "9m4e2mr0ui3e8a215n4g",
        Predictions: []Prediction{
            {
                Confidence: 0.69146144,
                Label:          "Lion",
                X:              109,
                Y:              682,
                Width:          77,
                Height:         25,
                Method:         "AI",
            },
            {
                Confidence:  0.7,
                Label:          "Penguin",
                X:              209,
                Y:              782,
                Width:          -13,
                Height:         -65,
                Method:         "AI",
            },
        },
    }

    got, err := ParseImageAnalysisResponse(data)
    if err != nil {
        t.Fatal(err)
    }

    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }

}

Links

https://bitfieldconsulting.com/golang/map-string-interface
https://jhall.io/posts/go-json-tricks-array-as-structs/
https://stackoverflow.com/questions/42377989/unmarshal-json-array-of-arrays-in-go