Understand how to use C libraries in Go, with CGO

You probably ever heard about CGO, and that Go can use shared libraries from your system to use the power of C. But how? I will explain the process, and you'll see that it's not that complicated – in fact, it's quite simple.

This tutorial is made for very beginners, but you need to have a bit of knowledge in C programming.

I wrote this tutorial for the simple reason that I find the official documentation, as well as the tutorials I've read, a little austere. For my part, I needed to start from a very simple situation, a real "hello world", and slowly move towards the use of a shared library. So this is my vision, my way of looking at things, which I offer you here.

What are shared libraries?

Shared libraries, also known as dynamic link libraries (DLL) on Windows or shared objects (SO) on Unix-based systems like Linux, are files containing compiled code that multiple programs can use simultaneously. Instead of having the code for a particular task duplicated in every program that needs to perform that task, the code is stored in a shared library. This way, programs can use the functions and procedures from these libraries without having to include the code directly in their files.

And Go can take advantage of this. Like Python or Rust, of course.

What does CGO?

It is a tool that allows calling C functions and using C libraries from Go code. It serves as the bridge between Go and C languages, enabling Go programs to incorporate existing C libraries and leverage existing C codebases.

Actually, it can do this:

• Passing Go functionalities to C but this is not what we will see today
• Calling C functions directly from a "comment" above the import "C" statement. Or from a .c file inside the project.
• Using C libraries to use already compiled functions and types in a shared library.

CGO is provided with Go, but you'll need a C compiler like GCC on Ming (for Windows).

CGO will use the "C" package where all C functions and variables are accessibles. It will then use the C compiler to make the link to your Go project.

Let's try to use C in Go!

First, just to understand what does CGO, we will call a C function that we will create.

Create a project, for example in ~/Projects/testCGO and inside this directory:

go mod init testcgo

Then create a main.go file and write this:

package main

// #include <stdio.h>
// void hello() {
//    printf("Hello from C")
//}
import "C"

func main(){
    // let's call it
    C.hello()
}

In your terminal, type go run . and see the result. Yes, it says "Hello from C".

But, what does CGO here?

Actually, CGO has used the comment above the import "C" statement and it shared the function in the "C" package.

That means that the hello() function, developped in C, is accessible as C.hello() in Go. The "C" package is like a namespace where C variables and function are accessed.

But, we can do this a bit prettier. Coding in comments can be useful if we don't have too many lines of C to do, but when the project becomes substantial, it can quickly become a bit annoying. So, let's use real C source files.

In the same directory, create the hello.c file:

#include <stdio.h>

void hello() {
    printf("Hello from C in another file")
}

And, a hello.h file to declare the function:

void hello();

Then, in your main.go file, replace the content to get this:

package main

// #include "hello.h"
import "C"

func main(){
    // let's call it
    C.hello()
}

This way, we're using the include statement, which is a C inclusion syntax. CGO has no problem:

go run main.go
Hello from C in another file

One more time, CGO takes the comments above the import "C" and because the #include statement is a valid C call, it compiles the C file without any problem.

We've just seen the base. Having C code on one side and a Go project on the other, we now know how to connect the two. But of course, there will be more restrictive things to manage.

How to work with C types?

Let's change the hello.c file to accept an argument and say hello to someone.

#include <stdio.h>

// say hello to the name
void hello(char* name) {
    printf("Hello %s\n", name);
}

Of course, change the header file to decralre the function:

void hello(char*);

Then, change the main.go file to now try to say hello to John:

package main

// #include "hello.h"
import "C"

func main() {
    C.hello("John")
}

This will fail...

./main.go:7:10: cannot use "John" (untyped string constant) as *_Ctype_char value in argument to (_Cfunc_hello)

That's something very important to keep in mind, we need to cast vars from and to Go and C.

Go ease the work with many types, like array, pointers, and strings. When you need to send or receive a variable from or to C, the types are not exactly the same. So we need to "cast" types. But, no panic, after a while you will do it naturally.

So, how to fix this?

We need to modify the Go string to char* type. We can use C.char type but that needs to manually allocate memory. Instead, there is a C.CString type which is a bit easier to use.

func main() {
    name := C.CString("Gopher")
    C.hello(name)
}

And now, it works!

This is the "complex" part of using CGO, you will need to convert, cast, manipulate the variables type to ensure that it will work.
And because it's C, there is no garbage collector for C variables, so you need to free memory when needed (using C.free())

So let's use a C library!

Now that we have seen how CGO can compile C code, let's try to tell it to link shared libraries.

For the example, I will use the very simple "libuuid".

You need to install the devel package of the library to get the header files. On Fedora, that was a simple sudo dnf install libuuid-devel command line.

To be able to generate a UUID, you need to read the documentation of the library (yes... RTFM...) - of course, I already did it and I can explain how to generate a UUID.

// In C:
// we need a uuid_t variable to initalize
uuid uuid_t;
// then we generate the random string. I use the random form
// but you can use other generate methods.
uuid_generate_random(uuid);
// To get a uuid string, we need to "unparse"
char uuid_str[37];
uuid_unparse_lower(uuid, uuid_str);
// In the uuid_str char*, there is a uuid

OK, so let's try.

We will include the uuid/uuid.h file, commonly in /usr/include on Linux, that CGO will find. And let's use the types and functions from this header file.

package main

// #include <uuid/uuid.h>
import "C"
import "fmt"

func main() {
    var uuid C.uuid_t
    var uuid_str *C.char
    uuid_str = (*C.char)(C.malloc(37))
    C.uuid_generate_random(uuid)
    C.uuid_unparse(uuid, uuid_str)
    fmt.Println(C.GoString(uuid_str))
}

This will fail...

The first problem, here, is that the typdedef doesn't work. We need to read the error to understand that, actually, uuid_t is a uchar*. But, not exactly... Actually, reading the header file, you'll see that it's a char[16].

Remember, in C, a char* is like a char[] (I'm grossly oversimplifying here).
But, libuuid declares the uuid_t with 16 chars of size, that means that, using pointer form, we need to allocate the memory with malloc.

So let's change the line to:

var uuid *C.uchar
uuid = (*C.uchar)(C.malloc(16))

You need to know a bit of C to work. Here, what I do is a simple C uuid = (uchar*)malloc(16) transposed with "C" package in Go.

Let's run one more time and...

/usr/lib/golang/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-645695464/000001.o: in function `_cgo_b1eb6bfc450b_Cfunc_uuid_generate_random':
/tmp/go-build/cgo-gcc-prolog:49: undefined reference to `uuid_generate_random'
/usr/bin/ld: /tmp/go-link-645695464/000001.o: in function `_cgo_b1eb6bfc450b_Cfunc_uuid_unparse':
/tmp/go-build/cgo-gcc-prolog:62: undefined reference to `uuid_unparse'
collect2: error: ld returned 1 exit status

That's now the time to use the "linker". Of course, we need to tel the compiler to use the libuuid.so shared library.

Understand the problem. Earlier, we used our own C source files that are compiled with CGO. But, now, we want to use "already" compiled sources to a ".so" library (or .dll for Windows). The header files are there to only provide function declaration (name, arguments and return types).

This is a common thing in C/C++ - that makes compilation very smart and fast, because we don't need to compile the libraries. We only "link" them.

To inform CGO to link the shared library, we can use a specific flags to the compiler. For libuuid it's a simple -luuid (understand -l uuid) to append. This will link libuuid.so to our binary. And Go proposes to specify these arguments inside the comments, as a #cgo: statement.

Above the inclusion of the header file, only append a special instruction:

// #cgo LDFLAGS: -luuid
// #include <uuid/uuid.h>
import "C"

And now it's OK

 go run  .
78137255-35a3-4f61-af7c-e04bf9eb513a

That's it, you have a UUID generated by a C shared library.

The entire source code is:

package main

// #cgo LDFLAGS: -luuid
// #include <uuid/uuid.h>
import "C"
import "fmt"

func main() {
    var uuid *C.uchar
    var uuid_str *C.char
    uuid = (*C.uchar)(C.malloc(16))
    uuid_str = (*C.char)(C.malloc(37))
    C.uuid_generate_random(uuid)
    C.uuid_unparse(uuid, uuid_str)
    fmt.Println(C.GoString(uuid_str))
}

It works very well, but is it practical? No...

Make it better

Calling C functions, with on-the-fly type casting, is impractical, unattractive and not at all easy to maintain.

Using C, the functions are way simpler to use:

uuid_t uuid;
uuid_generate_random(uuid);
char *str = malloc(37); // because 36 chars + \0
uuid_unparse_lower(uuid, str);
// and I can return "str" variable
// that contains a UUID

Then...

What do we really want? A function that gives us a UUID. So we're going to do something very practical:

code a function in C that will make our work easier, and just make sure we have access to it in our Go program.

OK, try:

package main

// #cgo LDFLAGS: -luuid
//
// #include <uuid/uuid.h>
// #include <stdlib.h>
//
// // create a uuid function in C to return a uuid char*
// char* _go_uuid() {
//   uuid_t uuid;
//   uuid_generate_random(uuid);
//   char *str = malloc(37);
//   uuid_unparse_lower(uuid, str);
//   return str;
// }
import "C"
import "fmt"

// uuid generates a UUID using the C shared library.
// It returns a Go string.
func uuid() string {
    return C.GoString(C._go_uuid())
}

func main() {
    // and now it's simple to use
    myuuid := uuid() // this is a go string now
    fmt.Println(myuuid)
}

Of course, we could create the _go_uuid() function in a C source file and create a .h file to declare our function. Then, include go_uuid.h.

What we did here is very common when we want to bind shared libraries to Go. We create some helper functions to cast types and to call C function without asking the user to use the C package by itself.

And this is how https://github.com/go-gst/go-gst, https://github.com/go-gl/glfw, and even https://fyne.io/ are using system libraries to propose a lot of functionalities.

Reminder

So, what you need to keep in mind when you want to use shared libraries:

• the "C" package gives access to C functions, types and variables
• you can include header files using comments above the import of "C" pacakge"
• you can provide LDFLAGS and CFLAGS to the compiler using the #cgo statement in comments
• you often need to cast types to Go types, or Go types to C
• you can create helpers in comments, in C, to ease the use of the librairies

Conclusion

C isn't the only language that lets you generate shared libraries. You can, of course, generate them in Go, Rust, Python, etc. But to date, it's C that's most widely used to generate libraries.

Having the possibility to use C, or calling C functions from a shared library opens Go to a wide range of powerful functionalities.

Obviously, we prefer libraries developed entirely in Go. This avoids dependence on a library that the user will have to intall on his system, or by sharing this library with him (in the form of .so or .dll). As Go is a language that normally uses static compilation, it's a "bit of a pity" to force the passage. But it's very useful in practice. For example, the very powerful Gstreamer library would be very complicated to recreate entirely in Go. It was created in C and works very well on many platforms. So here, having a dependency on this library is an excellent solution to open streaming sound and video to Go.

You'll need some knowledge of C to be able to link a library in Go. But you don't have to be a specialist. You just need to find the right variable cast, and create a helper function from time to time.

In any case, I hope that my article has opened the way for you, cleared up a few misunderstandings, and that you'll be able to use shared libraries with Go!

Comments

[Athul Muralidhar]

thank you thank you thank you! cgo was tough, this makes things so much better!
thx for the clear explanations and cheers!