Aniket Pal

Posted on May 15, 2023

Build Email Verifier With Go

#webdev #beginners #go #tutorial

The first step of cold mailing starts with getting the email address of the technical recruiter. It has been often said over the internet to randomly form email with the combination of first and last name of the recruiter.

Say, if you want to mail me the possible combinations can be aniket.pal@companymail.com, pal.aniket@companymail.com, paniket@companymail.com, aniketpal@comapanymail.com and related. Creating a humongous list of multiple permutation and combinations. To shorten the search space we will build an application to verify if the companymail.com is valid or not.

The purpose of the tutorial is to give you, the sheer understanding of the capabilities go-lang has. We won't be using any third party modules, with just the core modules of Go we will be building.

Why Choose Go? 🤨

The story goes, Google developers developed Golang while waiting for other languages to get compiled. Google developers had to completely rethink system development as a result of their displeasure with their toolset, which drove them to develop a lean, mean, and compiled solution that supports huge multithreading, concurrency, and performance under stress.

Every organisation looking at scale is leveraging Golang to build containerised microservices.

Moreover, Go is a great language for creating simple yet efficient web servers and web services. It provides a built-in HTTP package that contains utilities for quickly creating a web or file server.

Also, if you are interested in the Cloud Native Ecosystem, Go is the language you should start with. Incase, you have never developed a backend server with Go checkout Build Server With Go Under 10 minutes.

What we will build? 👨‍🚒

The small tool will check, if the email domain exists or not. The aim is to understand how to build backend and frontend with just using Go Lang. We will start with the backend server then shift to building the front end, while providing you a space to explore.

Building the Backend 🚪

We would be using the frontend of the application to get the domain we need to look for.

Defining package main and importing the required packages.



package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net"
    "net/http"
    "strings"

    "github.com/gorilla/mux"
)

If you are confused on how to install gorilla/mux and what does the package do, read Building a server with Go under 10 minutes. As of now, We will be building the REST APIs a little later. First, let us assume we get the domain as a string which we need to work upon.

Defining a handler, isValidDomain. The handler just takes one parameter say,domain of type string



func isValidDomain(domain string){
// controller code goes here
}

Let's start working with the controller function. Firstly, defining variables to check if the particular domain has MX Records, SPF Records and DMARC Records. If they have what are the records.



  var hasMX, hasSPF, hasDMARC bool 
  var spfRecord string
  var dmarcRecord  string

Now, it is the time to check on the internet for the data. The intensive in-house Go packages, get the job done with net/http package. Incase, you would have been using say ruby or nodejs you would have required to install further binaries. ```go

mxRecords,err := net.LookupMX(domain)

if err != nil{
    log.Printf("Error: %v\n",err)
}

if len(mxRecords)>0{
    hasMX = true 
}

We are using the net package to get the `mxRecords`, incase we don't get the data we get error. We handle the error in the following lines. To check if we have received mxRecords, we check if the length of the array is more than one, meaning if the array contains more than one element. 

```go


    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

Similar to mxRecords, we handle the data and error using := operator.

Remember, := is a declaration, whereas = is an assignment operator.

Post handling the error, we traverse the txtRecords slice, since we don't require the index we ignore it via _. Since, Golang doesn't permit having unused variables we use the underscore operator. When we loop over txtRecords, we check if version of the following record is spf1. If so, we mark hasSPF positive and store the value of record in spfRecord.



    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

Looking up for dmarcRecords is pretty much similar to txtRecords. We add _dmarc. in the prefix of the domain url, and use the net package to check if the corresponding records for dmarc exists or not. While traversing the dmarcRecords slice, we check if version of the record is DMARC1 we mark hasDMARC true and store the value of record in dmarcRecord.

To check if we are getting a values and our handler isValidDomain is working fine. Let's print the values we have till now.



fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

Compiling the isValidDomain script, we have.



func isValidDomain(domain string){
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }


    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)
}

Since, our function is working. Let's start working on building our REST APIs. Firstly, we will be defining 2 structs namely, DomainURL and DomainVar. One for decoding input and one for encoding the output.



type DomainURL struct{
    DomainURL string `string:"domainurl"`
}

type DomainVar struct{
    Domain string `json:"domain"`
    HasMX bool `json:"hasmx"`
    HasSPF bool `json:"haspf"`
    SpfRecord string `json:"spfrecord"`
    HasDMARC bool `json:"hasdmarc"`
    DmarcRecord string `json:"dmarcRecord"`
}

If you are a JavaScript developer struct is similar to ES6 class. Now, let us define a slice, which is similar to vectors in C++.



var domainVars []DomainVar

A slice is similar to an array, the difference is that when you want to use arrays in Golang you need to define the length. This is why we use a slice, we also tell it that it will contain posts. Over here domainVars is a slice of type DomainVar. Adding the route for POST request in the main function.

First creating a new request router. The router is the main router for our web application and will later be passed as parameter to the server. It will receive all HTTP connections and pass it on to the request handlers we will register on it. We create the router, in the main function. Post registering the router, let's define the endpoints using HandleFunction. We call HandleFunction by r.HandleFunc(...)



func main(){

    r := mux.NewRouter()
    r.HandleFunc("/form",formHandler).Methods("POST")

        fmt.Print("Starting server at port 8000\n")
        log.Fatal(http.ListenAndServe(":8000",r))
}

In HandleFunc we provide 2 parameters, firstly the route in which we want to see the magic and secondly we write the name of the particular controller function which performs the magic. The corresponding GET and POST has the regular meaning, to build the CRUD operation for the application just add PUT and DELETE according to the route.

The server port can be migrated to any value required.The script for POST Request Handler aka form Handler one step at a time.



func formHandler(w http.ResponseWriter, r *http.Request){
   w.Header().Set("Content-Type","application/json")
}

We define the the Header for the particular controller over here. We’re just setting the header “Content-Type” to “application/json”.



func formHandler(w http.ResponseWriter, r *http.Request){
   w.Header().Set("Content-Type","application/json")

   var domainUrl DomainURL
   json.NewDecoder(r.Body).Decode(&domainUrl)
}

Now, we pass the domain value in the isValidDomain function. We store all the in a domainVar. Post that we append the domainVar into the domainVars slice. Then we use the encoding package to encode all the domainVars data as well as returning it at the same line. Ultimately, sending in the POST request, fetching the data, storing in a struct, appending into the slice and then returning back the slice.



func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var domainUrl DomainURL
    json.NewDecoder(r.Body).Decode(&domainUrl)

    domainVar  := isValidDomain(domainUrl.DomainURL)
    domainVars = append(domainVars, domainVar)

    json.NewEncoder(w).Encode(domainVars)
}

Since, our route is configured let us modify our isValidDomain function such that it can we can get the required values for DomainVar struct.



func isValidDomain(domain string) DomainVar{
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

    var domainVar DomainVar
    domainVar.Domain = domain
    domainVar.HasMX = hasMX
    domainVar.HasSPF = hasSPF
    domainVar.SpfRecord = spfRecord
    domainVar.HasDMARC = hasDMARC
    domainVar.DmarcRecord = dmarcRecord

    return domainVar 
}

The modified function has a return type DomainVar which ultimately return the object of the same type. We create an instance type DomainVar and then start assigning values for each key. Finally return the object.

*The complete Backend Code as of now is: *



package main

import (
    "encoding/json"
    "fmt"
    "log"
    "net"
    "net/http"
    "strings"

    "github.com/gorilla/mux"
)

type DomainURL struct{
    DomainURL string `string:"domainurl"`
}

type DomainVar struct{
    Domain string `json:"domain"`
    HasMX bool `json:"hasmx"`
    HasSPF bool `json:"haspf"`
    SpfRecord string `json:"spfrecord"`
    HasDMARC bool `json:"hasdmarc"`
    DmarcRecord string `json:"dmarcRecord"`
}

var domainVars []DomainVar


func formHandler(w http.ResponseWriter, r *http.Request){
    w.Header().Set("Content-Type","application/json")

    var domainUrl DomainURL
    json.NewDecoder(r.Body).Decode(&domainUrl)

    domainVar  := isValidDomain(domainUrl.DomainURL)
    domainVars = append(domainVars, domainVar)

    json.NewEncoder(w).Encode(domainVars)
}

func isValidDomain(domain string) DomainVar{
    var hasMX, hasSPF, hasDMARC bool 
    var spfRecord string
    var dmarcRecord  string 

    mxRecords,err := net.LookupMX(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    if len(mxRecords)>0{
        hasMX = true 
    }

    txtRecords, err := net.LookupTXT(domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range txtRecords{
        if strings.HasPrefix(record,"v=spf1"){
            hasSPF = true 
            spfRecord = record
            break
        }
    }

    dmarcRecords, err := net.LookupTXT("_dmarc." + domain)

    if err != nil{
        log.Printf("Error: %v\n",err)
    }

    for _, record := range dmarcRecords{
        if strings.HasPrefix(record ,"v=DMARC1"){
            hasDMARC = true
            dmarcRecord = record 
            break
        }
    }

    fmt.Printf("domain=%v\n,hasMX=%v\n,hasSPF=%v\n,spfRecord=%v\n,hasDMARC=%v\n,dmarcRecord=%v\n",domain,hasMX,hasSPF,spfRecord,hasDMARC,dmarcRecord)

    var domainVar DomainVar
    domainVar.Domain = domain
    domainVar.HasMX = hasMX
    domainVar.HasSPF = hasSPF
    domainVar.SpfRecord = spfRecord
    domainVar.HasDMARC = hasDMARC
    domainVar.DmarcRecord = dmarcRecord

    return domainVar 
}


func main(){
    r := mux.NewRouter()

    r.HandleFunc("/form",formHandler).Methods("POST")


    fmt.Print("Starting server at port 8000\n")
    log.Fatal(http.ListenAndServe(":8000",r))
}

Building the Frontend 🎨

We won't be actually focusing on beautifying the application rather our focus would be on getting things done. For the frontend, we would require a package go-fiber. Although, the frontend could have been developed without installing any additional package, the reason for using go-fiber is to understand how to install and work with external packages.

Let's start with creating the directory and changing our working directory to it.



mkdir verifier-frontend && cd verifier-frontend

Initialising the main.go file for the frontend



touch main.go

Running the command, creates an empty file in our working directory over here which is verifier-frontend.

We need to define a package for the Go file. Since this will be the main file we have, we add the package main.



package main

The above line is the first line of the program.

Now, let's import all the necessary packages we require to build our application 🛍️



import (
    "log"

    "github.com/gofiber/fiber/v2"
)

If, I would have been you and read all the imports, I would have been pretty much confused. But, trust me once you read the blog I can bet you would get a clear idea why we included the following packages.

Creating the go.mod file which will store all the necessary packages required, it is similar to package.json.



go mod init github.com/Aniket762/namaste-go/verifier-front

To ensure that the go.mod file matches the source code in the module, we run



go mod tidy

All the imported packages except for one is already present with the binary file you executed while installing Go. So, let's install Gorilla Mux.



go get github.com/gofiber/fiber/v2

If you are a JavaScript developer, the following command is pretty much similar to npm install

Now, we are all set to start building our frontend. Let's get the code from Go Fiber's official documentation, run and understand it.



package main

import (
    "log"

    "github.com/gofiber/fiber/v2"
)

func main() {
    app := fiber.New()

    app.Get("/", func (c *fiber.Ctx) error {
        return c.SendString("Hello From Verifier's Frontend 👋")
    })

    log.Fatal(app.Listen(":3000"))
}

Post importing the packages, we write the main function. The main function is the first which gets started when we compile and execute the program. We create an instance, app of type fiber. We shorthand c for the context.

The HTTP request and response are held in the context, which is represented by the Ctx struct. It provides methods for the request's body, HTTP headers, arguments, and query string.

Let's run the server at port 3000. You can change the port according to your convenience. Now, let us run the server and check if everything is in sync. Switch to your terminal and execute go run main.go

You should get this on your terminal which shows the port we are running into, number of handlers, process IDs and number of processes we are running. The following data is of real use when you shift to building more complex applications. Navigate to localhost:3000 or which ever port you have written.

Outro 💚

The application we have developed is a prototype and not at all production ready. The only purpose of the blog was to give you the exposure on how to build a full stack application with native go packages. You can research further and build a production ready application.

Now, just like any other tutorial let us end the tutorial with a task. We have developed the REST APIs using gorilla/mux and frontend starter with go-fiber. Try build the POST route for the frontend. Incase you aren't able to build I would be soon posting an article on building frontend with Go-lang. All the best for building the complete frontend. Incase, you don't miss when I publish how to build frontend using go-fiber follow me on my socials ^-^

Incase you have developed the application or have anything to discuss under the sun feel free to get in touch with me on LinkedIn or Twitter 💖