DEV Community: Dhanush Gopinath

Concurrent HTTP downloads using Go

Dhanush Gopinath — Mon, 05 Feb 2018 13:37:03 +0000

Recently for an implementation of feature in Geektrust, we had to download multiple images from the internet and use it. This had to be an efficient implementation. This post is a brief explanation of the simple implementation done using goroutines and channels.

Go has an http package which has simple APIs to do HTTP operations. For e.g. if you want to download a page all you need to do is this.

resp, err := http.Get("http://example.com/")
if err != nil {
    // handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)

Given below is the implementation of downloading a file from a public URL and using it as bytes.

For downloading multiple files concurrently you can use the help of goroutines and channels.

In the above code snippet, we initialise two channels, one for sending the file downloaded as []byte type and another for sending any errors that happened while downloading.

We then loop though all the urls and call the downloadFile method for each of them in a goroutine. These goroutines are executed concurrently and so the for loop is finished processing in no time. Inside the goroutine each file is downloaded and we get back bytes and error.

If there is an error in the download, then the error is pushed to the error channel and nil value is pushed to the done channel. If there is no error then the bytes are pushed to the done channel and a nil entry is pushed to the error channel.

An array of byte arrays is declared to collect the downloaded files back. We again loop through as many times as there are URLs to collect the byte array back from the done channel.

bytesArray = append(bytesArray, <-done)  
if err := <-errch; err != nil {   
   errStr = errStr + " " + err.Error()  
}

These two lines inside the for loop wait for the done and the error channel to receive data. Once the data has arrived, it increments the loop count. Bytes are appended to the array of bytes while error string is appended, if there are any errors.

You have now downloaded the files concurrently and they are available as an array of bytes. All just using simple Go code.

This article was first published on my personal tech blog.

Sending HTML emails using templates in Golang

Dhanush Gopinath — Wed, 21 Jun 2017 07:05:22 +0000

In the Geektrust application (built on NodeJS) we send a lot of emails that are auto-generated at runtime using HTML Templates. In a new application, which we have developed in Go, we needed the same feature.

This post is a short read about reading an HTML template file and sending email using Go’s html/template and net/smtp packages.

The code is as given below.

package main

import (
    "bytes"
    "fmt"
    "html/template"
    "net/smtp"
)

var auth smtp.Auth

func main() {
    auth = smtp.PlainAuth("", "iamwho@whoami.com", "password", "smtp.gmail.com")
    templateData := struct {
        Name string
        URL  string
    }{
        Name: "Dhanush",
        URL:  "http://geektrust.in",
    }
    r := NewRequest([]string{"junk@junk.com"}, "Hello Junk!", "Hello, World!")
    err := r.ParseTemplate("template.html", templateData)
    if err != nil {
        ok, _ := r.SendEmail()
        fmt.Println(ok)
    }
}

//Request struct
type Request struct {
    from    string
    to      []string
    subject string
    body    string
}

func NewRequest(to []string, subject, body string) *Request {
    return &Request{
        to:      to,
        subject: subject,
        body:    body,
    }
}

func (r *Request) SendEmail() (bool, error) {
    mime := "MIME-version: 1.0;\nContent-Type: text/plain; charset=\"UTF-8\";\n\n"
    subject := "Subject: " + r.subject + "!\n"
    msg := []byte(subject + mime + "\n" + r.body)
    addr := "smtp.gmail.com:587"

    if err := smtp.SendMail(addr, auth, "dhanush@geektrust.in", r.to, msg); err != nil {
        return false, err
    }
    return true, nil
}

func (r *Request) ParseTemplate(templateFileName string, data interface{}) error {
    t, err := template.ParseFiles(templateFileName)
    if err != nil {
        return err
    }
    buf := new(bytes.Buffer)
    if err = t.Execute(buf, data); err != nil {
        return err
    }
    r.body = buf.String()
    return nil
}

In this I encapsulate the smtp request in a struct Request. It contains basic things like To, Subject, Body. This template file has the data placeholders which will be replaced with actual data values by Go’s html/template package Execute method.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>

</head>

<body>
<p>
    Hello {{.Name}}
    <a href="{{.URL}}">Confirm email address</a>
</p>

</body>

</html>

The template data is a struct which has Name & URL as the field values. It is initialised with values and passed to the NewRequest method to create an object of Request.

templateData := struct {
    Name string
    URL string
 }{
    Name: "Dhanush",
    URL: "http://geektrust.in",
 }

Once the instance of Request is created, then ParseTemplate method is called on it. This method receives a template filename and template data. It parses the template file and executes it with the template data supplied. A bytes.Buffer is passed to the Execute method as io.Writer so that we get back the HTML string with the data replaced.This HTML string is then set as the Email Body.

The SendEmail method sets the MIME encoding as text/html and calls smtp package’s SendMail method to send the email. When the email is sent successfully the SendEmail method returns a true value.

Notes:

If you return the Request in ParseTemplate method, instead of the error, then we make the call in one line. ok,err := NewRequest([]string{“junk@junk.com”}, “Hello Junk!”, “Hello, World!”, “template.html”, templateData).ParseTemplate().SendEmail() But then any error coming out of ParseTemplate may have to be tracked inside the method.
Read the package documentation of html/template to understand how Go replaces the HTML with actual data.
Thanks to Jijesh Mohan and Navaneeth K. N in making the Go code more idiomatic, than it was before :)
This post was first published in my personal blog

Hi, I'm Dhanush Gopinath

Dhanush Gopinath — Sat, 18 Mar 2017 07:46:18 +0000

I have been coding for close to 14 years.

You can find me on Twitter as @dhanushgopinath

I live in Bangalore, the Garden City and Silicon Valley of India, but I am a Malayali from Calicut - where Vasco da Gama set foot in 1498 - in North Kerala.

I started Geektrust 2 years back along with 2 friends and am currently the CTO there.

I mostly program in Go, Javascript & Python for the last 2 years. Before that I was a Java programmer for over 10 years. But I have been in love with Go now.

I am currently learning more about Machine Learning & NLP and I would love to learn Elixir as my next Programming language.

Apart from programming I am a voracious reader with Gabriel Garcia Marquez, Jose Saramago & M.T. Vasudevan Nair being my favourite writers

Nice to meet you.