DEV Community

Toby Chui
Toby Chui

Posted on • Edited on

2 3

Moving Go application to cloud VM: Gzip Middleware

Recently I was working on an ArozOS update and someone suggested that he want to deploy ArozOS (which is a web desktop platform) to a cloud VM and he needs to minimize the bandwidth used in serving media and web template files.

How Gzip Middleware Works

After some research, I decided to add a layer of "gzip middleware" in between the mux / router and the response writer layer. If you try to illustrate the new layer in a diagram, it will looks like this
2021-03-07_12-21-53

Implementation

After knowing what I need, I wrote an implementation of it using Go Module architecture. The following is the full code extracted from my ArozOS project gzip middleware module.

package gzipmiddleware
import (
"compress/gzip"
"io"
"io/ioutil"
"net/http"
"strings"
"sync"
)
/*
This module handles gzip on http.Handler and http.HandleFunc
author: tobychui
*/
var gzPool = sync.Pool{
New: func() interface{} {
w := gzip.NewWriter(ioutil.Discard)
gzip.NewWriterLevel(w, gzip.BestCompression)
return w
},
}
type gzipResponseWriter struct {
io.Writer
http.ResponseWriter
}
func (w *gzipResponseWriter) WriteHeader(status int) {
w.Header().Del("Content-Length")
w.ResponseWriter.WriteHeader(status)
}
func (w *gzipResponseWriter) Write(b []byte) (int, error) {
return w.Writer.Write(b)
}
/*
Compresstion function for http.FileServer
*/
func Compress(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
h.ServeHTTP(w, r)
return
}
w.Header().Set("Content-Encoding", "gzip")
gz := gzPool.Get().(*gzip.Writer)
defer gzPool.Put(gz)
gz.Reset(w)
defer gz.Close()
h.ServeHTTP(&gzipResponseWriter{ResponseWriter: w, Writer: gz}, r)
})
}
type gzipFuncResponseWriter struct {
io.Writer
http.ResponseWriter
}
func (w gzipFuncResponseWriter) Write(b []byte) (int, error) {
return w.Writer.Write(b)
}
/*
Compress Function for http.HandleFunc
*/
func CompressFunc(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
fn(w, r)
return
}
w.Header().Set("Content-Encoding", "gzip")
gz := gzip.NewWriter(w)
defer gz.Close()
gzr := gzipFuncResponseWriter{Writer: gz, ResponseWriter: w}
fn(gzr, r)
}
}

See the whole system implementation here

GitHub logo tobychui / arozos

General purposed Web Desktop Operating Platform / OS for Raspberry Pis, Now written in Go!

Image

IMPORTANT NOTES

The current arozos is still under intense development. System structure might change at any time. Please only develop on the current existing ArOZ Gateway Interface (AGI) JavaScript Interface or standard HTML webapps with ao_module.js endpoints.

Features

User Interface

  • Web Desktop Interface (Better than Synology DSM)
  • Ubuntu remix Windows style startup menu and task bars
  • Clean and easy to use File Manager (Support drag drop, upload etc)
  • Simplistic System Setting Menu
  • No-bull-shit module naming scheme

Networking

  • FTP Server
  • WebDAV Server
  • UPnP Port Forwarding
  • Samba (Supported via 3rd party sub-services)
  • WiFi Management (Support wpa_supplicant for Rpi or nmcli for Armbian)

File / Disk Management

  • Mount / Format Disk Utilities (support NTFS, EXT4 and more!)
  • Virtual File System Architecture
  • File Sharing (Similar to Google Drive)
  • Basic File Operations with Real-time Progress (Copy / Cut / Paste / New File or Folder etc)

Others

  • Require as little as 512MB system memory and…

To use it, simpliy call it with your standard http.HandleFunc and http.Handle as follows

if *enable_gzip {
    http.HandleFunc("/media/", gzipmiddleware.CompressFunc(serverMedia))
}else{
    http.HandleFunc("/media/", serverMedia)
}
Enter fullscreen mode Exit fullscreen mode
fs := http.FileServer(http.Dir("./web"))
if *enable_gzip {
    //Gzip enabled. Always serve with gzip if header exists
    http.Handle("/", gzipmiddleware.Compress(fs))
} else {
    //Normal file server without gzip
    http.Handle("/", fs)
}
Enter fullscreen mode Exit fullscreen mode

Results

The following is the compression results of the ArozOS desktop template. You can see the gzip middleware helps to save a around 160KB per request. If you are running it on a cloud server, this might save you a few bucks in long run.

2021-02-16_13-24-51
(Top: With gzip middleware enabled; Bottom: With gzip middleware disabled)

Well, basically that is it. Hope you find it useful :)

Reinvent your career. Join DEV.

It takes one minute and is worth it for your career.

Get started

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay