Integrating C libraries into Go (Example with libbz2)

Hi, my name is Walid, a backend developer who’s currently learning Go and sharing my journey by writing about it along the way.
Resource :

• The Go Programming Language book by Alan A. A. Donovan & Brian W. Kernighan
• Matt Holiday go course

---

In this tutorial, we will demonstrate how to integrate Go with the libbzip2 compression library using cgo. We will keep the C code in a separate file and call it from Go to compress a file.

---

Prerequisites

Before we begin, ensure you have the following installed:

• libbzip2 Development Library:

  • On Debian-based systems:

  sudo apt install libbz2-dev
  

---

Project Structure

To keep things organized, we will separate our Go and C code into different files. Create the following directory structure:

project-directory/
├── main.go
├── bz2compress.c
└── bz2compress.h

---

Writing the C Code

1. Create bz2compress.h

This header file declares the functions we will implement in bz2compress.c.

#ifndef BZ2COMPRESS_H
#define BZ2COMPRESS_H

#include <bzlib.h>

int bz2compress(bz_stream *strm, int action, char *in, unsigned int *inlen, char *out, unsigned int *outlen);
bz_stream* bz2alloc(void);
void bz2free(bz_stream* strm);

#endif // BZ2COMPRESS_H

2. Create bz2compress.c

This file implements the functions declared in the header.

#include "bz2compress.h"
#include <stdlib.h>

int bz2compress(bz_stream *strm, int action, char *in, unsigned int *inlen, char *out, unsigned int *outlen) {
    strm->next_in = in;
    strm->avail_in = *inlen;
    strm->next_out = out;
    strm->avail_out = *outlen;
    int ret = BZ2_bzCompress(strm, action);
    *inlen -= strm->avail_in;
    *outlen -= strm->avail_out;
    strm->next_in = strm->next_out = NULL;
    return ret;
}

bz_stream* bz2alloc(void) {
    return (bz_stream*)calloc(1, sizeof(bz_stream));
}

void bz2free(bz_stream* strm) {
    free(strm);
}

---

Writing the Go Code

Now, let's write the Go code to use these C functions.

3. Create main.go

package main

/*
#cgo LDFLAGS: -lbz2
#include "bz2compress.h"
*/
import "C"
import (
    "fmt"
    "io"
    "os"
    "unsafe"
)

func main() {
    inputFile := "input.txt"
    outputFile := "output.bz2"

    inFile, err := os.Open(inputFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error opening input file: %v\n", err)
        os.Exit(1)
    }
    defer inFile.Close()

    outFile, err := os.Create(outputFile)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error creating output file: %v\n", err)
        os.Exit(1)
    }
    defer outFile.Close()

    const bufferSize = 64 * 1024
    inBuffer := make([]byte, bufferSize)
    outBuffer := make([]byte, bufferSize)

    strm := C.bz2alloc()
    defer C.bz2free(strm)

    if ret := C.BZ2_bzCompressInit(strm, 9, 0, 0); ret != C.BZ_OK {
        fmt.Fprintf(os.Stderr, "BZ2_bzCompressInit failed: %d\n", ret)
        os.Exit(1)
    }
    defer C.BZ2_bzCompressEnd(strm)

    for {
        inLen, err := inFile.Read(inBuffer)
        if err != nil && err != io.EOF {
            fmt.Fprintf(os.Stderr, "Error reading input file: %v\n", err)
            os.Exit(1)
        }

        if inLen == 0 {
            break
        }

        inLenC := C.uint(inLen)
        outLenC := C.uint(bufferSize)

        ret := C.bz2compress(strm, C.BZ_RUN,
            (*C.char)(unsafe.Pointer(&inBuffer[0])), &inLenC,
            (*C.char)(unsafe.Pointer(&outBuffer[0])), &outLenC)

        if ret != C.BZ_RUN_OK {
            fmt.Fprintf(os.Stderr, "BZ2_bzCompress failed: %d\n", ret)
            os.Exit(1)
        }

        if _, err := outFile.Write(outBuffer[:outLenC]); err != nil {
            fmt.Fprintf(os.Stderr, "Error writing to output file: %v\n", err)
            os.Exit(1)
        }
    }

    fmt.Println("Compression completed successfully.")
}

---

Building and Running the Application

1. Initialize the Go Module

   go mod init project-directory

1. Build the Project

   go build

1. Run the Application

   ./project-directory

Ensure input.txt exists before running the program. The compressed output will be saved as output.bz2.

---

Conclusion

By following this tutorial, you have successfully integrated C code with Go using cgo. Keeping the C code separate improves maintainability and allows better modularity. This approach can be extended to use other C libraries in Go applications efficiently.

If you have any questions or suggestions, feel free to leave a comment below!