DEV Community

Promise Femi
Promise Femi

Posted on

Handing file upload with Go

It has been one hell of a journey learning go, it been some-worth difficult as I was very used to the loose nature of JavaScript and PHP (languages I have been writing for about 3 years). GO's strict nature is something I have just been coming into terms with in the last few months and by all means, loving it. It is a great language.

The reason for this post is to try to explain file upload in GO through the eyes of an amateur, so pardon me if my writing is very verbose as i will be trying to explain this as i would to myself. so let's dig in.

We would be handling file uploading as part of the form body as well as handling multiple files.

First let's create the boilerplate, create a folder and create a new main.go file.

package main

func uploadHandler(w http.ResponseWriter, r *http.Request){

}

func main(){
        http.HandleFunc("/upload", uploadHandler)
        log.Fatal(http.ListenAndServe(":9000", nil))
}
Enter fullscreen mode Exit fullscreen mode

next, let us create a simple index.html file where we would be making our submissions from

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Go file upload</title>
</head>

<body>

    <form action="/upload" method="post">
        <input type="file" name="image" accept="image/*">
        <button type="submit">Submit</button>
    </form>

</body>

</html>
Enter fullscreen mode Exit fullscreen mode

lets also serve the index.html file in main.go

package main

func uploadHandler(w http.ResponseWriter, r *http.Request){

}
func indexHandler(res http.ResponseWriter, r *http.Request) {
    res.Header().Add("Content-Type", "text/html")
    http.ServeFile(res, r, "index.html")
}

func main(){
http.HandleFunc("/", indexHandler)
        http.HandleFunc("/upload", uploadHandler)
        log.Fatal(http.ListenAndServe(":9000", nil))
}
Enter fullscreen mode Exit fullscreen mode

I do recommend that when testing with the index.html file you add enctype="multipart/form-data" since browsers will not submit the file without this in the form tag and that would cause an error, or you could use postman until we get to parsing multipart forms.

next, we move forward to handling the form in the request, first, we must parse the form

func uploadHandler(w http.ResponseWriter, r *http.Request){
            // this neccessary else you will not be able to access other properties needed to get the file
// this method returns an err           
        err:=r.ParseForm()
// always handle error
     if err != nil{
// of course we should use a better error message
http.Error(w, "Unable to parse form", http.StatusInternalServerError)
return
}

}

Enter fullscreen mode Exit fullscreen mode

the reason why parsing the form is that we need to call the r.FormFile() method and this method as well as the r.FormValue() methods are unavailable until the form is parsed.

in this case, we will be using the r.FormFile() method, this method returns three arguments:

First argument is the multipart.File type.
the second argument is the multipart.FIleHeader type these arguments contain the meta details of the file such as its name, size and the file itself."
the third and final argument is an error.

with that being said let jump in and take a look at the added lines of code

func uploadHandler(w http.ResponseWriter, r *http.Request) {

    // this neccessary else you will not be able to access other properties needed to get the file
    // this method returns an err           
            err:=r.ParseForm()
    // always handle error
     if err != nil{
            http.Error(w, "Unable to parse form",  http.StatusInternalServerError)
            return
        }

    file, fileHeader, err := r.FormFile("image") 
// get the file by entering its form name.
// handle errors as neccessary
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
//postpone closing the file until the function is complete
defer file.Close()
Enter fullscreen mode Exit fullscreen mode

after the previous step, it is now time to create the file on the local server, we are going to be using the current time as the name of the file and extracting the file extension with the filepath.Ext() method.

func uploadHandler(w http.ResponseWriter, r *http.Request) {
/*
reduced for brevity
*/
// create file on the local server, filepath.Ext() will get the extension out of the filename
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(fileHeader.Filename))
//handle errors as neccessary
if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

Enter fullscreen mode Exit fullscreen mode

Now we can copy the uploaded file into the newly created file.

/*
reduced for brevity
*/
// create file on the local server, filepath.Ext() will get the extension out of the filename
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(fileHeader.Filename))
//handle errors as neccessary
if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

// use the io.Copy() method to copy bytes to the localbyte from the uploaded file,
// this method returns the number of bytes copied and an error, we would be ignoring the bytes
_, err = io.Copy(localfile, file)
//handle errors as neccessary
if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

//after which we can simply respond back to the client
fmt.Fprint(w, "File upload successful")
}
Enter fullscreen mode Exit fullscreen mode

and that's it, that is all that is needed to upload a file.

Parsing a multipart form

if the form submitted is a multipart form, parsing the form is slightly different from a normal form but only by one line.

//change this line
err:=r.ParseForm()
//to, r.ParseMultipartForm() takes an argument (maxMemory), maxMemory indicates how much of the file size should be kept in memory, while any extra will be kept in a temp folder on the disc
err:=r.ParseMultipartForm(300)

Enter fullscreen mode Exit fullscreen mode

Uploading multiple files

Uploading multiple files is just as simple as uploading a single file with the major difference being how you handle getting the files and looping through them, lets take a look.

// create a new function for uploading multiple files
func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
        r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb
}
Enter fullscreen mode Exit fullscreen mode

let us also handle that in the main.go

func main() {

    http.HandleFunc("/", indexHandler)
    http.HandleFunc("/upload", uploadHandler)
    http.HandleFunc("/uploadmultiple", uploadMultipleFiles)

    http.ListenAndServe(":9000", nil)
}
Enter fullscreen mode Exit fullscreen mode

after parsing the form, we would now be able to get the files with the r.MultipartForm.File["images"] slice.

func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
        r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb

//we get the slice of files
files:= r.MultipartForm.File["images"]

}
Enter fullscreen mode Exit fullscreen mode

we need to loop through the files to be able to upload each of the files just like we did before, with the only difference being that we would not be working with an actual multipart.File type but with a multipart.FileHeader type instead. but not to worry the multipart.FileHeader also contains the file itself.

func uploadMultipleFiles(w http.ResponseWriter, r *http.Request){
        r.ParseMultipartForm(1024 * 1024 *10) //maxMemory if 10mb

//we get the slice of files
files:= r.MultipartForm.File["images"]
// looping through the files to upload each file individually
    for _, file := range files {

        uploadFile, err := file.Open()
        // handle error as neccessary
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }

        // postpone file closure until the function is complete
        defer uploadFile.Close()

        //create a new localfile
        localfile, err := os.Create(time.Now().UTC().String() + filepath.Ext(file.Filename))
        //handle errors as neccessary
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }

        _, err = io.Copy(localfile, uploadFile)
        //handle errors as neccessary
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
    }

    //after which we can simply respond back to the client
    fmt.Fprint(w, "File upload successful")
}
Enter fullscreen mode Exit fullscreen mode

Extras

Setting a maximum file upload size

in a case where you want to restrict the file upload size, we would simply equate the file size in bytes:

// add where neccessary in your code
const MAX_UPLOAD_SIZE int64 = 1024 * 1024 * 10 //maximum of 10mb per file
// if this was for a single file upload it would be fileHeader.Size, because it is the
// multipart.FileHeader type that has Size a property
if file.Size > MAX_UPLOAD_SIZE{
http.Error(w, "file sizes cannot be bigger than 10mb", http.StatusBadRequest)
            return
}
Enter fullscreen mode Exit fullscreen mode

Tracking file upload progress

Following up with file upload so as to check if the file upload progress is a bit tricky but not complicated. First, we need to create a new struct type whose property would be the total size of the file (TotalBytesToRead) and how many bytes have been read (TotalBytesRead). the struct also needs to satisfy the io.Writer interface, OK let's see some code.

type Progress struct{
    TotalBytesToRead int64
    TotalBytesRead int64
}

// create the write method so satisfy the i0.Writer interface
func (p *Progress) Write(pb []byte) (int, error) {
    n := len(pb) //length of the file read so far
    // set error to nil since no error will be handled

    p.TotalBytesRead = int64(n)

    if p.TotalBytesRead == p.TotalBytesToRead {
        fmt.Println("Done")
        return n, nil
    }
//this will  be printed on your terminals
    fmt.Printf("File upload still in progress -- %d\n", p.TotalBytesRead)

            return n, nil
    }

// now in use with the file upload 

func uploadHandler(w http.ResponseWriter, r *http.Request) {

    // this neccessary else you will not be able to access other properties needed to get the file
    // this method returns an err           
            err:=r.ParseForm()
    // always handle error
     if err != nil{
            http.Error(w, "Unable to parse form", http.StatusInternalServerError)
            return  
        }

    file, fileHeader, err := r.FormFile("image") 
// get the file by entering its form name.
// handle errors as neccessary
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
//postpone closing the file until the function is complete
defer file.Close()

//create a new localfile 
localfile, err := os.Create(time.Now().UTC().String()+filepath.Ext(file.Filename))
//handle errors as neccessary
if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

// Create a new instance of progress
progress := &Progress{
TotalBytesToRead : fileHeader.Size,
}

//instead of coppying directly we would use the io.TeeReader() method
// this method will allow us to write the amount of bytes read into the progress.Write method
_, err := io.Copy(localfile, io.TeeReader(file, progress))
//handle errors as neccessary
if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

}
Enter fullscreen mode Exit fullscreen mode

and that is it, that is all that is needed to track file upload progress.

at this point, I hope you are now a master in the arts of file uploading and all of its caveats. thank you for reading and i hope this was helpful. please follow me.

Complete code can be found here

Top comments (0)