DEV Community

loading...
Cover image for Zig Makes Go Cross Compilation Just Work

Zig Makes Go Cross Compilation Just Work

kristoff profile image Loris Cro ・6 min read

For the last couple of months I worked on a redesign of https://ziglang.org. Among other things, the site was ported to Hugo, a popular static site generator written in Go. Everything went smoothly, but I did encounter a snag when setting up the deploy pipeline: I could not build Hugo for x86_64 Linux from my Apple Silicon Mac mini!

Go cross-compilation failed

The failed build.

How could this be?

Go does have the ability to compile a project for another platform, you just need to specify GOOS and GOARCH when running go build, like I did in the screenshot above. The problem is the remaining environment variable: CGO_ENABLED, which caused the build command to fail.

Hugo C extensions

It just so happens that Hugo can be built with or without a set of C extensions used to manipulate CSS and other assets. If you want the C extensions (like in my case), you need to enable cgo, a piece of the Go toolchain that handles compilation and linking of C code.

Pesky C code

Compiling C code has always been a bit of a nuisance, and especially so when it comes to cross compilation. If you search for how to cross compile cgo you will find a long list of suffering and hopelessness. This is what Dave Cheney replied to one of such questions on Stack Overflow:

Dave Cheney recommends giving up on cross compiling cgo (he was not wrong at the time)

Well, a few years plus a lot of collective effort later, I'm happy to show you how to cross compile trivially :)

Say hello to Zig

Zig is a new programming language that has no runtime, no macros, a radical compile-time metaprogramming system, and seamless C interoperability. You can even import C header files directly and immediately use all the definitions in your Zig code, without needing any glue / bindgen.

Even better, Zig is a full-fledged C/C++ cross compiler that leverages LLVM. The crucial detail here is what Zig includes to make cross compilation possible: Zig bundles standard libraries for all major platforms (GNU libc, musl libc, ...), an advanced artifact caching system, and it has a flag-compatible interface for both clang and gcc.

This means that Zig is a dependency-free, in-place replacement for your current C/C++ compiler that allows cross compilation out-of-the-box. Just download a Zig tarball, extract it somewhere, and boom: you can now cross compile to your heart's content.

Let's see how to use Zig from Go.

How to use Zig to cross compile Hugo (with C extensions)

First of all, you need to download Zig. You can either get a tarball as mentioned above, or have your favorite package manager install everything for you. You can even find Zig in Homebrew (Mac) and Chocolatey (Windows).

You also need to make sure zig is present in your PATH, so that you can call the compiler from any directory. If you're not sure how to do it, check out the Getting Started guide. Package managers should take care of PATH for you, if you decide to go that route.

To test if you have setup Zig correctly, run zig version in a terminal, it should reply with something similar (i.e. it should not error out, but the version might be different of course):

0.7.1
Enter fullscreen mode Exit fullscreen mode

Invoking Zig from Go

We're finally at the climax! How hard is it to call into Zig when compiling a cgo project?

If you have Go version 1.17 or above (NOTE: at the moment of writing this version is not out yet), then you only have to tell Go to use Zig to compile C/C++ code.

If you want to cross compile for x86_64 Linux, for example, all you need to do is add CC="zig cc -target x86_64-linux" CXX="zig c++ -target x86_64-linux" to the list of env variables when invoking go build. In the case of Hugo, this is the complete command line:

CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zig cc -target x86_64-linux" CXX="zig c++ -target x86_64-linux" go build --tags extended
Enter fullscreen mode Exit fullscreen mode

That's it! Really. Trivial cross compilation indeed.

Important notes

As you probably noted, we've repeated the cross compilation target multiple times.

Unfortunately, Go doesn't provide this information to the C/C++ compiler, so it's up to us to provide that little bit of glue. This means that if you want a different target, you will have to change both Zig invocations and the GOOS/GOARCH variables.

Another important detail is that Zig calls x86_64 what Go calls amd64. That's the most notable difference in naming conventions, so keep that in mind.

Finally, you may be interested in knowing that Zig can also accept a third option when specifying the target architecture: the libc ABI.

For Windows, you want gnu (e.g. x86_64-windows-gnu) because that will use Zig's bundled MinGW-w64 instead of trying to find an MSVC installation. Note: there's a problem with targeting Windows, tracked in this issue (downstream issue).

For Linux, you probably want musl (e.g. x86_64-linux-musl) because your resulting binary will be statically linked and thus work on all Linux distributions. However, if you prefer to interact with the system glibc, such as on Ubuntu, you can specify gnu (e.g. x86_64-linux-gnu).

The Zig language reference contains the full list of supported targets.

For older versions of Go

For versions of Go lower than 1.17, you need a workaround for this bug (relative GH issue).

This workaround consists of 2 bash scripts that wrap the two Zig commands into single-argument commands (it might seem silly, but that's what the bug is about).

Here are the steps:

1. Create the scripts

zcc

#!/bin/sh
ZIG_LOCAL_CACHE_DIR="$HOME/tmp" zig cc -target x86_64-linux $@
Enter fullscreen mode Exit fullscreen mode

zxx

#!/bin/sh
ZIG_LOCAL_CACHE_DIR="$HOME/tmp" zig c++ -target x86_64-linux $@
Enter fullscreen mode Exit fullscreen mode

2. Make the scripts executable

$ chmod +x zcc zxx
Enter fullscreen mode Exit fullscreen mode

3. Add the scripts to PATH

If you don't know how to do it, it's the same procedure explained in the Getting Started guide: you want to add to PATH the directory containing zcc and zxx.

4. Use the scripts as your C/C++ compiler

Just specify CC="zcc" CXX="zxx" when building and you're good to go! Here's the full command line for Hugo:

CGO_ENABLED=1 GOOS=linux GOARCH=amd64 CC="zcc" CXX="zxx" go build --tags extended
Enter fullscreen mode Exit fullscreen mode

This is what I did in my case and, well, it just worked.
The successful build

The successful build.

Conclusion

I think Andrew (the creator of Zig) captured the conclusion perfectly in this Tweet.

Is this solution Go-only?

This should be easy to infer, but to be absolutely clear: no, this is not a feature designed specifically for Go.

Zig can be used as a C/C++ cross compiler directly or from other toolchains.

Zig can also be used by cc-rs, a Rust crate used for shelling out to a C/C++ compiler, for example.

If you want a more detailed explanation read this blog post by Andrew.

Couldn't you just download a prebuilt Hugo executable?

I have my own small fork of Hugo where I added a custom integration with zig-doctest, a tool that both tests and renders to html the real output of most of the code snippets present on the website.

GitHub logo kristoff-it / zig-doctest

A tool for testing snippets of code, useful for websites and books that talk about Zig.

In other words, I had to build my own executable.
Originally the CI on GitHub would build Hugo every run, but that took 4 mins out of a 5 mins total runtime. After this change, we can now deploy in about 1 minute, with most of the time spent testing Zig code snippets, as should be.

Oh no, it doesn't work!

You tried but got blasted with errors anyway?

There are two possibilities: either you did something wrong, or we did (i.e. there's a bug somewhere or a particular C/C++ feature that's not yet supported).

Here's how to fix that:

  1. Join a Zig Community and ask for help. People will be able to help you fix common mistakes. If this doesn't work, goto step 2.
  2. Open an Issue on GitHub. Make sure to explain in detail your setup and share the full error message you received. We'll do our best to help you, especially if you did your due diligence with step 1.



Now it's your turn to get out there and cross compile for great justice!

Discussion (3)

pic
Editor guide
Collapse
nektro profile image
Meghan (she/her)

Woah! The more I learn about what Zig is capable of, the more it continues to amaze me.

Collapse
fahmifan profile image
fahmi irfan

This is nice

Collapse
fr3fou profile image
fr3fou!! 🎀

this is quite nice!