Email is one of the media that is often used to send notifications to end-users. Knowing the system architecture used to send email notifications is very important so that it can run well and quickly. One system architecture scheme that can be used is a worker pool, so that the API can respond to requests quickly while the task of sending emails is done concurrently.
Get to know the Worker Pool
In a worker pool, incoming tasks are added to the queue, while being worked on by a concurrent pool of workers.
We can analogize it to sending a package to a logistics service. First, the package is registered and weighed to get an air bill. Then the package that has been given an index will be put into the queue of goods while waiting to be sent to the destination by the courier.
Here is a picture of the workflow of the worker pool:
Techstack
Techstack is an important tool in building systems. Here is the techstack that will be used:
- Golang 1.22 version: can use higher or lower version
- smtp gmail: a free smtp server provided for every gmail account
- Postman: client to test our API
Guide to Setup Gmail Smtp
- Visit https://myaccount.google.com/ don't forget to login with your account
- Visit the 2-step verification page. It may be called 'Two Step Authentication' in English.
- Enter your google account credentials. Then enable two factor authentication.
- Scroll down and then Go to the Application Password page. In English it may be called 'Application Password'.
- Enter the application name and copy the application password, then paste it into the .env file.
Let's Get Started on the Project
- Project structure
Explanation:
- main.go: starting point to start the application
- .env: file for storing environment data such as credentials. Store the application password in this file.
- .gitignore: To prevent commits to important files such as .env. Note: the .env file should not be committed as it stores important credentials.
- /mail: folder or package for sending emails via smtp
- /jobs: package for creating worker pools
- 2. Let's Start Coding a) Project initialization
- Perform project declaration
Type:
go mod init <project-name> - download the required dependencies. type this command on cmd
go get github.com/gin-gonic/gin, github.com/joho/godotenv
Notes: For the gin package, you can use other web framework packages such as Fiber, or the native net/http package.
b) Create a Dto package to standardize the request format
package dto
type Body struct {
Sender string `json:"sender" binding:"required"`
Receiver string `json:"receiver" binding:"required"`
Subject string `json:"subject" binding:"required"`
Message string `json:"message"`
}
type Response struct {
Message string `json:"message"`
Data map[string]interface{} `json:"data"`
}
c) Create Package mail to send emails.
package mail
import (
"crypto/tls"
"fmt"
"go-notify/dto"
"net/smtp"
"os"
)
const port = "587" //25 or 587
const smtpDomain = "smtp.gmail.com"
var M *Mail
type Mail struct {
Auth smtp.Auth
hostname string
From string
}
func NewMail() *Mail {
return &Mail{
hostname: smtpDomain,
}
}
func (m *Mail) SetAuth() {
username := os.Getenv("GOOGLE_MAIL")
pass := os.Getenv("GOOGLE_APP_PASSWORD")
m.From = username
m.Auth = smtp.PlainAuth("", username, pass, m.hostname)
}
func (m *Mail) SendMail(content dto.Body) error {
addr := m.hostname + ":" + port
// Create a TLS configuration
tlsConfig := &tls.Config{
InsecureSkipVerify: false, // Set to true only if needed for testing
ServerName: "smtp.gmail.com",
}
// client instance
client, err := smtp.Dial(addr)
if err != nil {
return err
}
defer client.Close()
// Initiate TLS handshake
if err = client.StartTLS(tlsConfig); err != nil {
return err
}
// Authenticate with the server
if err = client.Auth(m.Auth); err != nil {
return ErrAuth
}
// Set the sender and recipient
// if err = client.Mail(m.From); err != nil {
// return ErrSender
// }
if err = client.Mail(content.Sender); err != nil {
return ErrSender
}
if err = client.Rcpt(content.Receiver); err != nil {
return ErrRecipient
}
// Write the email body
writer, err := client.Data()
if err != nil {
return err
}
_, err = fmt.Fprintf(writer, "Subject: %s\r\n\r\n %s.", content.Subject, content.Message)
if err != nil {
return err
}
err = writer.Close()
if err != nil {
return err
}
// Send the QUIT command and close the connection
if err = client.Quit(); err != nil {
return err
}
return nil
}
d) Create package jobs for the workerpool
package jobs
import (
"go-notify/dto"
"go-notify/mail"
"log"
"sync"
)
type queue struct {
Body dto.Body
retry uint
}
type WorkerPool struct {
Queue chan queue
Worker uint
}
func NewWorkerPool(worker uint, size int) *WorkerPool {
return &WorkerPool{
Queue: make(chan queue, size),
Worker: worker,
}
}
func (wp *WorkerPool) Add(request dto.Body, rty uint) {
q := queue{
retry: rty,
Body: request,
}
wp.Queue <- q
}
func (wp *WorkerPool) Work(wg *sync.WaitGroup) {
defer wg.Done()
log.Println("initializing worker")
for job := range wp.Queue {
log.Println("processing job ", job.Body.Receiver)
err := mail.M.SendMail(job.Body)
if err != nil {
log.Println(err)
// if sender or recepient not found, omit proccess and continue to next queue
if err == mail.ErrRecipient || err == mail.ErrSender {
continue
}
// retry
// pass proccess and continue to next queue
if job.retry > 3 {
log.Println("this job has been retried more than 3 times")
continue
}
job.retry++
wp.Add(job.Body, job.retry)
continue
}
log.Println("success send mail to: ", job.Body.Receiver)
}
}
func (wp *WorkerPool) Do(wg *sync.WaitGroup) {
for i := 0; i < int(wp.Worker); i++ {
log.Println("creating worker: ", i+1)
go wp.Work(wg)
}
}
e) Create router, controller, and package initialization in main.go
package main
import (
"go-notify/dto"
"go-notify/jobs"
"go-notify/mail"
"log"
"net/http"
"sync"
"github.com/gin-gonic/gin"
"github.com/joho/godotenv"
// _ "github.com/joho/godotenv"
)
func main() {
var err error
err = godotenv.Load(".env")
if err != nil {
panic(err)
}
mail.M = mail.NewMail()
mail.M.SetAuth()
var wg = &sync.WaitGroup{}
// You can use other settings for the number of workers and queues.
// Experiment and adjust to your device
wp := jobs.NewWorkerPool(5, 25)
wg.Add(int(wp.Worker))
wp.Do(wg)
r := gin.Default()
r.POST("/api/mail", func(ctx *gin.Context) {
var req dto.Body
if err := ctx.ShouldBindJSON(&req); err != nil {
ctx.JSON(http.StatusBadRequest, dto.Response{
Message: "error while binding body into struct",
Data: map[string]interface{}{
"error": err.Error(),
},
})
return
}
log.Printf("binding sender:%s receiver:%s\n", req.Sender, req.Receiver)
wp.Add(req, 0)
ctx.JSON(200, gin.H{
"message": "ok",
})
})
r.POST("/api/mail/v2", func(ctx *gin.Context) {
var req dto.Body
if err := ctx.ShouldBindJSON(&req); err != nil {
ctx.JSON(http.StatusBadRequest, dto.Response{
Message: "error while binding body into struct",
Data: map[string]interface{}{
"error": err.Error(),
},
})
return
}
err = mail.M.SendMail(req)
if err != nil {
ctx.JSON(http.StatusBadRequest, dto.Response{
Message: "failed to send email",
Data: map[string]interface{}{
"error": err.Error(),
},
})
return
}
ctx.JSON(200, gin.H{
"message": "ok",
})
})
r.Run(":8080")
wg.Wait()
}
d) Try to run App
- type in your cmd:
go run main.goit will display the console as follows:
- then hit the API
POST http://localhost:8080/api/mailin Postman
- Look at the console, it will show the following result:
And here is the inbox of the message sent to the recipient:
Don't forget to Follow:
- Linkedln: BAGAS SEBASTIAN
- Web Portofolio: portofolio
- Github: github
Top comments (0)