DEV Community: Michele Caci

A Go-cus after GopherConEU 2023

Michele Caci — Mon, 17 Jul 2023 07:15:47 +0000

Because if there has been a bin-go, there can be a go-cus (or a focus on Go).

Some time has passed since GopherConEU 2023, held in June 27th to 29th, and I wish to take this occasion to share with you my experience of these amazing days and my thoughts about the areas where Go has shown to focus more using the content of what was shared in the various talks and the chatter with the various Gophers.

TL;DR: GopherConEU is one of the best places to have an idea of what is going on about Go, especially going in person, as this makes you learn not only from the content of the conference but also from meeting and talking to the Gophers around you.

An overview

Welcome back

For those who don't know yet what GopherConEU is, it is the European conference for all topics related to Go.

Every edition of GopherConEU, this being the 6th, has been organized in a different location, but since last year it was localized in Berlin at the Festsaal Kreuzberg.

For GopherConEU 2023 I was able to participate to the workshop and the two days of talks. I was not able to arrive in time to join the social events at the beginning of the talk but it did not prevent me to meet very nice people during the conference days. I sure felt the friendly atmosphere that sourrounds the Go community during the conference and everyone was eager to meet and to exchange not only about Go but also their passions and interests.

To share my experience and my idea of what stood out during the GopherConEU this year, I'm going to focus on some specific topics:

A focus on the Go community
Go-ing to next level
The importance of security

A focus on the Go community

Go is primarily made by the Go team. During GopherConEU this sear there were several talks from the Go team members and other Google software engineers who made some highlights about few new features and tools, like the talks of Jonathan Amsterdam about the new slog package coming in go v1.21, the one about service weaver, made by Robert Grandl, a framework recently released by Google to be able to modularize a Go binary into services that can be easily deployed as microservices and the lightning talk from Robert Burke who made me excited to go get go version v1.21 and experiment the updates on generics.

But what stood out the most during these three days was the fact that the community itself is growing more and more, adopting Go to experiment and create all sorts of applications and pacakges and, most of all, sharing its experiences with the rest of the community.

This was highlighted not only by the keynote given by Cameron Balahan, the product lead of Go, about the state of Go, but also on the concrete examples of some on the talks given during the conference.

Take for example the talk about the tips to reduce memory consumption on Go by Yiscah Levy Silas where she shared her experience about memory allocations and exploit this knowledge to make the right choices and reduce the memory footprint of your applications. In another example you can take the talk from Drishti Jain where she explained which libraries and concepts we need to know to develop a 2D Game in Go, which, by the way, I agree with her that is a fun way to learn Go. In fact, I actually started a long time ago a very small console game wallrush, with the excuse of practising Go with a fun project.

Drishti Jain on how to develop a 2D Game in Go

There are so many other things I could mention but this could get extremely long, so we'll move to the next topic, but not before a quick mention to the TinyGlobo 2 launch.

Go-ing to next level

This topic is the trickiest to describe but I'll make my best to convey what I took out from this with these two examples.

The first one is related to the workshop of Bill Kennedy about Ultimate Services in Kubernetes where he shared several tips and trick to both create an application and deploy it in Kubernetes and to take well known Go concepts and take them to the next level.
Some of these ideas are the following:

Have a deploy-first mentality: prepare your deployment with all the configuration you need and a minimal code to support it. Once your minimal application is deployed in the cluster you can concentrate on the business logic
Think about the package as an API, which creates firewalls between code sections as it can only be accessed via its exported symbols. Also if a package is an API, it serves by providing functionalities, not by containing them, which is a very important distinction between a functional package and a package that just serves as a bin containing things we might forget
Start by accepting data for what it is and then discover what it does (concrete type vs interface).
Consider that errors are handled when acknowledged or treated in some way, for example by logging them. Once handled, the error should not be propagated any further.
Prefer to not deal with errors inside an http.Handler but make sure that you create an Handler with a function that returns an error so that it can be handled in every situation, e.g. by defining a type Handler func(ctx context.Context, w http.ResponseWriter, r *http.Request) error

Bill Kennedy at the Ultimate Services workshop

These are just few examples where I noticed something clicking in my head, I really tried to see through the examples given during the workshop and I'm convinced that I need to give a try myself to understand it better and assimilate these concepts as well.

The second example, I think of, is the talk done by Julien Cretel about how to create and use functional options during the creation of values from a structure. This idea is not new because it exploits a lot the concept in Go of functions as first class citizens, but it is also not easy to put in place as it requires some understanding in how to use functions to configure the values of a field. The fact of going from func New(/* parameters with values to assign to MyStruct fields*/) *MyStruct to func New(opts ...func(*MyStruct)) *MyStruct is just the first step to help create values in a cleaner way that goes beyond simple "Constructor functions" or "Configuration structures". In his talk, however Julien didn't just explain how to create functional options but also when and how to use them to make sure that we can get the best of their implementation.

The importance of security

Security is one of the pillar of today's focus when developping an application, as more often than not, attacks uncover the vulnerabilities that affect it or the libraries on which it depends on.

Go is not undervaluing the security efforts and this was highlighted in both keynotes: the aformentioned one by Cameron Balahan about the the state of Go and the one focusing on vulnerability management in Go by Julie Qiu.

If we focus first on the keynote from Julie Qiu, the manager of the security team in Go, it was important to hear about the meaning of vulnerability and how they can affect the supply chain (by actually using a very tasty cake analogy 😄🍰). To address this the go team worked on several layers:

tooling
data
documentation

The tooling has seen the introduction of the vulncheck command, just recently released, that helps to identify security issues inside the dependencies of a Go module with an effort to reduce the noise and convey only the important information in order to fix the most urgent security issue. The vulncheck tool also sees it's first integration int he CI thanks to the appropriate Github Action that has been implemented to include it in a CI build for a Go application.

The data is taken care thanks to the vulnerability database, which provides a comprehensive source of information about known vulnerabilities in public Go modules, which in turn are shown in the documentation that will display a sign if there has been a vulnerability detected inside the package page in .

Julie Qiu on how to manage vulnerabilities

These information go to support the story behind the state of Go which is not just a language that is growing with time but also a full consolidating ecosystem that is designed to be a productive and production-ready platform. For this reason it is not only important that Go is easy to learn to get onboarded, but also that it comes with its own tools to perform many operations, like testing, benchmarks, module management, fuzzing and vulnerability checking (to name a few) that make the base for the creation of stable, secure, reliable and efficient applications with Go.

A personal note

GopherConEU 2023 was not just for me a place where I heard interesting talks and topics about my favorite programming language.

It was for sure an occasion to meet a lot of nice Gophers, old and new, chat with them and share our experiences. Be it with the Go team or with the other Gophers it was nice to share my projects like wallrush and the Go workshop I presented at RivieraDEV or my experience to help the organization of RivieraDEV and at the same time hear about the others and see what they were working on both on the professional side and on the free time.

Speaking of personal projects one stood out during the conference and it was gopherconeu23img. Why was it so special? It's because, it's a CLI, totally written in Go, that creates GIFs in ASCII art from an input text and, most importantly, it was used to create some of the GIFs that were included in the slides of this year's edition of GopherConEU. 😍

Here are some examples:

Another thing I want to mention is Giuseppe Scaramuzzino's lightning talk Unleashing Desktop App Development with Go and Fyne which was great, informative and very well received by the audience. I'm proud he was selected to perform his talk and to have played a small part on his decision to submit this talk. Great job! 😎

Giuseppe's Lightning talk about Fyne

Parting thoughts

The idea of this article was not to be exactly a report on the talks of GopherConEU 2023 as there are some good reports out there that highlight well the content of the talks. The idea was to highlight some thoughts on the state of Go and its community after meeting all the Gophers in person and hearing the content of the talks proposed during the conference.

It's a bit of a pity that I might not be able to attend to the other editions of GopherCon (a baby 👶 is coming, so unless I'm talking I prefer to attend remotely where possible) this year but I hope that these thoughts will be confirmed and some new ones will get the spotlight.

GopherCon EU was not just an occasion for sharing and learning, it was also a moment of distention and entertainement, I'm thinking especially of the social mixer after the first day of conference and the live episode of Go Time podcast with a session of the "Gopher Say" quiz game. If you haven't had the chance to participate, the talks will be made available in YouTube, so at some point you'll be able to access the content as well.

I will close again by thanking everyone that made this edition of GopherConEU and sharing some more pictures of this two days marathon.

You can find me up on linkedin, github and twitter at @goMicheleCaci or here on dev.to @mcaci and I'm looking forward to hearing your comments.

This was Michele and thanks for reading!

The queue for the workshop day

Panel with the Go team

Some of the GopherConEU Swag

My GoLab 2022 experience

Michele Caci — Fri, 14 Oct 2022 21:01:56 +0000

Laptop courtesy of my sweet wife

Last week the 2022 edition of GoLab 2022 happened and I wish to share with you my experience for the two days of conference that I attended and share my view of the different talks that were presented.

This edition was special as the equivalent conference for Rust, RustLab, was also organized at the same moment, but more on this soon.

TL;DR: The idea to have GoLab and RustLab together was nice as it encouraged people with different view and experiences to meet and share. On the GoLab side there were a lot of interesting talks and people to speak with, I really enjoyed it!

Let's start

Golab 2022 was composed of several days of events: the first day was the one for the workshops and the two following days were the conference days. I just participated to the two days of talks but it was already a good way to meet as many Gophers and share ideas and knowledge about Go.

This edition of GoLab was organized by Develer just outside the city of Florence in Italy.

Let's start

About the conference

I want to share the content of these two days of conference by building around a few concepts that I think well summarize the content of what was shared during the conference:

Going beyond the simplicity
Go outside of Go
Striving towards security

Going beyond the surface

Some of the talks of these two days focused on going behind the surface of Go either by showing some obscure language behaviors or by exploring the standard library. One of the biggest highlight of this section is that most of the time the Go documentation provide explanations about these behaviors or well explains the packages even though they are not consulted until needed.

This was reflected, for example, in the talk about the things you didn't know about Go by Roberto Clapis where he gave a deep view on the internals of Go and some very unexpected behaviors in a very entertaining way. He showed us also that the explanation for these unexpected behaviors can often be found in the Go documentation which provides a precious resource to consult at any time of need.

In this context I would also highlight the talks of Boston Cartwright about the sql package and the driver pattern of Federico Paolinelli where he gave some practical tips on which constructs and syntax elements to use to write Go code that is simple to read and maintain. From these talks, I took away the contrast between the simplicity that Go offers to developers with the complexity inside the language that makes it possible to be simple to use.

This in a way is summed up in a talk given by Rob Pike about the complexity that makes Go simple to use: "Simplicity is Complicated".

Go outside of Go

One of the recurrent topics during these two days where the usage of Go outside of the plain language itself and especially I think of two specific cases: the usage of Go in microservices and the integration of C and Go with the cgo feature.

On the microservice side there was a nice talk given by Adelina Simion and Arthur Kondas about communication between microservices where they gave useful tips about when and how to go from a monolith to microservices while also sending very positive vibes from the stage.

Technical problems are no problems

On the cgo side I want to highlight two talks: one from Loris Cro about dealing with cross-compilation difficulties, that the usage of cgo brings, using the Zig language and the other from Jason Dellaluce and Leonardo Grasso about how to extend Falco, a Kubernetes threat detection engine, which is written in C++, with plugins written in Go, explaining the challenges of integrating cgo in both C and Go.

It was very interesting to me to see how Go can introduce its way through various projects and architectures making it a viable language in contexts where other mainstream languages, like C++, Java or Python, used to be the main languages of choice.

Striving towards security

For sure one of the big topic of the conference was the security around Go, the main argument being that security is built-in Go thanks to the standard library for example with the crypto package or thanks to the fuzzying feature that was recently introduced with Go version 1.18.

Filippo speaking about the crypto package

In this context I want to highlight the talks of Filippo Valsorda where he explained how is security managed and tested inside Go's crypto package, the talk from Sagar Sonwane with an entertaining and clear introduction to fuzzying and the talk from Daniel Marti where he explained us what techniques of defensive programming we can make use of to defend from unforeseen, and potentially dangerous, usages of our code.

This is not the first time that security in Go is highlighted in a conference but it was good to see the message about security reinforced with more insights and practical information that us gophers can use.

Parting thoughts

I could go on speaking and detailing about all the talks that were given as they were all interesting and insightful but I would never end writing my thoughts.

In the end, GoLab was not just an occasion for sharing and learning, it was also a moment of entertainment and meeting with new people in a friendly environment and If you haven't had the chance to participate, the best way to learn more about the content will be to watch the talks when they will be made available on YouTube.

I will close again by thanking everyone that made this edition of Golab possible and sharing some more pictures of the conference.

You can find me up on twitter @nikiforos_frees or here on dev.to @mcaci and I'm looking forward to hearing your comments.

This was Michele, thanks for reading!

At the conference venue

Daniel Marti explaining defensive programming

Closing the conference with a twilight over Florence

My GopherConEU 2022 experience

Michele Caci — Wed, 10 Aug 2022 05:30:44 +0000

A week has already passed since GopherConEU 2022, held in July 28th to 31st, and I wish to share with you my experience of these amazing conference days and my take of the content of what was said in the various talks.

TL;DR: it was great to be back in person, meet new Gophers, hearing a lot of interesting talks and I was very happy to be speaking to a live audience for the first time.

Let's start

GopherConEU 2022 was composed of several days: a community event, the workshop day and a two days run of talks. My participation was limited to the two days of talks but I sure felt the great energy and friendly atmosphere that sourrounds the Go community. Everyone was eager to exchange and learn about Go, either on specific topics or on the general trends.

Every edition of GopherConEU, this being the 5th, has been organized in a different location, Iceland and the Canary islands to name a few; this year's edition was organized in Berlin in the Festsaal Kreuzberg.

Let's start

A summary of two days

I'm going to build the resume of I took away from the talks of these two-days conference by discussing around these elements:

Embracing the simplicity
A focus on memory
Practical uses of Go
The Go team

Embracing the simplicity

Some of the talks of these two days highlighted the simplicity of the Go language by looking at the experiences that the speakers had with their work and with some references to the UNIX philosophy of building modular, simple and transparent software.

This was reflected, for example, in the talk about CLIs in Go by Marian Montagnino where she gave a lot of insights on how to make a CLI that feels closer to the user than to the machine; in the talk about making the service easier to extend and refactor by transforming it into a platform by Konrad Reiche. In this one in particular the focus was in making the distinction between eventual complexity, that is intrinsic to the problem we want to solve, and accidental complexity, which is the one we introduce while solving a a problem, and his experience showed us a way to reduce the accidental complexity; and as well another in the talk given by Egon Elbre that gave really useful tips about how (and when) to use concurrency, by showing patterns and constructs that are safer and simpler to use when introducing concurrency in our applications.

And last but not least, the talk by Yarden Laifenfeld where she shared her experience of migrating an application from an OOP language (Java) to Go and we reinforced the message that when going from a language to Go (or any other language) it is never a plain translation, but an embracement of Go values and features, because they are too different from one language to another and this holds very true when thinking of Java and Go.

This to me highlights the experiences of the Go community when striving in making things simple, as said in the Go Proverb "Clear is better than clever".

A focus on memory

A lot of discussions were made around the memory and resources usage: from the talk about the new memory limit feature coming with Go 1.19 explained to us by Michael Knyszek to the entertaining story about a feud between Bill Kennedy and a man named Tyler about the memory allocations done by their algorithms and some practical tips how to use pprof to investigate the performances and reduce the resources usage and when it's really worth doing it.

Bill Kennedy on memory profiling

This was very interesting in a context where we want take care of our consumptions to see how to check what we consume and where we can do better and in which situations it is important (or not) to do it.

Practical uses of Go

Many of the talks were also about how the community has used Go in its own projects.

Some of the highlights in this regard were:

The talk about the usage of TinyGo to check the environment inside the hen house of Donia Chaiehloudj where she made use of TinyGo to create an application embedded in an integrated circuit that took some measures and sent them to Grafana.
The talk by Robert Burke about how to use Apache Beam, now supporting Go, to perform stream processing in our applications, to consider the difference between event time and processing time and especially to focus on windows of event times instead of the event time itself.
The talk by Anna-Katharina Wikert where she explained us how to follow the vulnerable parts of our code from the source, e.g. sensitive data, to a potential sink, e.g. a log.Println statement without any sanitization process, using the go-flow-levee tool.

I could go on speaking and detailing about all the talks that were given as they were all interesting and insightful but I would never end my speech so let's move to...

The Go team

There were few interactions with the Go team, besides the talk of Michael about memory limits, that were centered on a panel with the Go team where people could ask questions to some members of the Go team, Than Macintosh, Cameron Balahan, Michael Knyszek and on some reflections shared by Cameron on how Go has become the Go we love today.

Some of the highilghts I took away were around the history of Go, how it grew by embracing the community via the feedbacks from the surveys and the github issues and how it maintained its importance by embracing the enterprise by brainstorming around ideas that could help enterprises adopt Go more easily and how it keeps evolving besides language features, e.g. in its toolset.

How Go became its best self

"No is the the default answer" is a sentence shared by the Go team to convey in a very effective way the fact that Go's values are simplicity and stability; this is in comparison to other languages that may add new features more frequently and make them sometimes more complex and challenging to embrace.

A last interesting point, shared by the Go team, was again about security as the evolution of Go keeps growing around functionalites and tools that help for instance with the dependency checks, e.g. the tampering check on the go.sum in Go modules and the introduction of fuzzing to help unit test our application to find potentially dangerous edge cases.

A personal note

GopherConEU 2022 was not just for me a place where I heard interesting talks and topics about my favorite language.

First of all it was an occasion to meet a lot of nice Gophers from all over Europe, exchange and share our experences with them. There I met, for instance, nice people from Arduino, people from Celltech using Go to work on the creation and testing of batteries and other nice companies using Go for many different projects from Finance and Payment (Upvest and Form3) to Food Delivery (Delivery Hero), Data Federation (Hygraph) and Marketing and App promotion (Remerge).

Then, this edition of GopherConEU was special to me because it was my first experience of talking in public to a live audience. My talk was the second lightning talk where I did a small live coding session to show the power of the standard library when it comes to take the first steps to develop a web application.

My first live talk 😍

First live talk + lightning talk + live coding = a lot of challenge.

And a challenge was what I wanted to propose, to myself, to the organizers and to the audience, if selected. I must admit I was a bit nervous when I proposed it, but as I got accepted it was time to take on this challenge!

And so it happened!

For this I must thank all the organization of the GopherConEU, from Natalie to Donna and everyone involved to give me this opportunity and honor and also a big thanks to the co-hosts Mat Ryer and V Korbes that supported and encouraged me before going on the stage.

Should you want to know more about this talk you can check my websvc repo.

Parting thoughts

We sure were! 😁

GopherCon EU was not just an occasion for sharing and learning, it was also a moment of distention and entertainement, I'm thinking especially of the improvised talks given by Mat Ryer and V Korbes on subjects given by the attendees (testing and doing code review with people you don't like) and a Live Episode of Go Time podcast with the "Gopher Say" Game.

Again, there are so many things that are worth mentioning but time is running out. If you haven't had the chance to participate to GopherCon EU, the talks will be made available in YouTube at some point you'll be able to do it soon.

I will close again by thanking everyone that made this edition of GopherConEU and sharing some more pictures of this two days marathon.

You can find me up on twitter @nikiforos_frees or here on dev.to @mcaci and I'm looking forward to hearing your comments.

This was Michele, thanks for reading!

EDIT: The video for my lightning talk has been published! I hope you enjoy it!

TIL: How to listen to different servers in Go (using select)

Michele Caci — Mon, 25 Apr 2022 17:48:09 +0000

Today I learned how to listen to different servers in Go by refactoring an existing codebase of mine. The goal of this refactoring was to correctly use select statement instead of creating an "artificial" one.

This is the starting point: I'm omitting most of the code to best highlight the "artificial" select that was present at the beginning of my refactoring.

func Start(/* options here*/) error {
    errChan := make(chan error)
    go func() {
        c := make(chan os.Signal, 1)
        signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
        errChan <- fmt.Errorf("%s", <-c)
    }()
    // [...]
    go startHTTPSrv(errChan)
    // [...]
    go startGRPCSrv(errChan)
    // [...]
    return <-errChan
}

func startHTTPSrv(errChan chan<-error) { // the param list is simplified to focus only on the err chan
    errChan <- http.ListenAndServe(/* http.ListenAndServe params */)
}

func startGRPCSrv(errChan chan<-error) { // the param list is simplified to focus only on the err chan
    errChan <- gRPCServer.Serve(/* gRPCServer.Serve params */)
}

Since the same errChan is used everywhere, any of the goroutines that ends up returning an error, be it the HTTP server, the GRPC one or the SIGINT/SIGTERM listener, will make the Start func exit.

This is actually the behaviour of a select statement; which is why I started transforming it and this is how it looks now.

func Start(o *Opts) error {
    select {
    case err := <-startHTTPSrv(newSrvData(o.HTTPAddr)):
        return err
    case err := <-startGRPCSrv(newSrvData(o.GRPCAddr)):
        return err
    case err := <-handleSigTerm():
        return err
    }
}

func startHTTPSrv(/* params */)  <-chan error{ // the param list is simplified to focus only on the err chan
    // [...]
    errChan := make(chan error)
    go func() {
        errChan <- http.ListenAndServe(/* http.ListenAndServe params */)
    }()
    return errChan
}

func startGRPCSrv((/* params */) <-chan error { // the param list is simplified to focus only on the err chan
    // [...]
    errChan := make(chan error)
    go func() {
        // [...]
        errChan <- gRPCServer.Serve(/* gRPCServer.Serve params */)
    }()
    return errChan
}

func handleSigTerm() <-chan error {
    // [...]
    errChan := make(chan error)
    go func() {
        c := make(chan os.Signal, 1)
        signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
        errChan <- fmt.Errorf("%s", <-c)
    }()
    return errChan
}

At this point, the usage of the select statement is able to show the clear distinction between the different goroutines run and wait for the first one to exit to return from Start.

Besides the change in the Start func, the other functions have also been changed to return a receiver chan of errors instead of a plain error in order to be comfortably used inside the select statement.

There are two positive aspects in this refactor:

It gives a good showcase for the usage of the select statement to listen to different processes/servers.
It shows a way to transition from an "artificial" select statement to an idiomatic one.

I hope this was helpful, thanks a lot for your time reading it!

p.s. Here be dragons! A.k.a the github reference for the commit I did with the complete change that inspired this TIL.

TIL: Go range on time.Tick for regular tasks

Michele Caci — Thu, 26 Aug 2021 15:46:02 +0000

Today I learned how to simply repeat a task over time with regularity in Go:

for range time.Tick(1 * time.Second) {
    fmt.Println("Hello, playground") // any task here
}

Here's a playground example in action: https://play.golang.org/p/RUdgii5EMx7

TIL: Find alone and find within loops in a shell

Michele Caci — Mon, 19 Apr 2021 16:51:00 +0000

Today I learned a situation in which is good to use a for loop in a shell in moments when the -exec flag alone is difficult to use.

The find command in any case is very powerful and can assist in many shell tasks. Usually when executing commands on a set of file the find command, together with the -exec expression in the find $PATH -exec $COMMAND {} \; instruction, is very useful to write short and meaningful operations like

find ./ -name $PATTERN -exec cat {} \;

to print the content of a list of files denoted by the PATTERN variable.

However, it is not very useful when the situation gets complex like in this scenario.

Suppose we have 2 files:

file1 with content

-l
file2 with content

-a

And we want to run ls with the options found in these files.

In this context the command

find ./ -name "file*" -exec ls $(cat {}) \;

would not work (the error would be "cat: {}: No such file or directory").

In this situation a classic for range loop will help us achieve this goal. As in this example

for i in $(find ./ -name "file*"); do ls $(cat $i); done

Despite being very unconventional uses of those instructions, I propose these examples to draw a line on where the find command manages to do the job alone and where it is more helpful to include its resuls it in a for range loop.

Hope this was helpful, thanks a lot for your time reading it!

TIL: The smart difference between infinite for loops that work and those that break

Michele Caci — Sun, 21 Mar 2021 18:33:41 +0000

Consider this Go function with an infinite for loop:

func exampleA() int {
    for {
        /* do something */
        return 1
    }
}

Take a moment to reflect on this question: Does this code break at compilation time?

If you think: "well yes, there is a missing return before the function closes; take as example the following code where a break is introduced before the return. How could this function work?"; I would agree that your statement and example is correct and that exampleB would indeed get a missing return message from the compiler

func exampleB() int {
    for {
        break
        return 1
    }
}

However you may very well be surprised to discover that, if you leave the function exampleA unaltered, without the missing return at the end, and test it somewhere (for example in Go's Playground it will compile and execute without complaining.

How is it possible?

An infinite for statement without a break inside is considered by the compiler as a terminating statement, a statement that is located at the end of a list of statement and that blocks the execution of any further statements within the same block.

Which means that the following example also works despite the absence of any return statement.

func exampleC() int {
    for {}
}

The infinite for loop with no breaks is not the only terminating statement present in Go; for more details please consult this section of the golang reference spec.

Hope this was helpful, thanks a lot for your time reading it!

TIL: A (maybe) unexpected usage of the -count flag in Go tests

Michele Caci — Fri, 12 Feb 2021 18:32:49 +0000

Normally when you add the -count $N flag to the go test command the result is that any tests and/or benchmarks that would be run are run $N times as we can see in the following run.

go test -v -run 1 -bench 1 -count 3
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
=== RUN   Test1Sort
-------- PASS: Test1Sort (0.00s)
goos: linux
goarch: amd64
Benchmark1Sort
Benchmark1Sort-12          10370            113657 ns/op
Benchmark1Sort-12          10488            114596 ns/op
Benchmark1Sort-12           8942            115987 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        5.422s

Just a side note, the 1 put as argument of -run and -bench are regular expressions to identify respectively which tests and which benchmarks to run; for more informations on benchmarks, you can read this article.

Getting back to the situation described in the title, let's look at this situation where we have a test run that is cached after the first time it is run.

$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.017s
$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        (cached)
$ go test -run 1 .
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        (cached)

Tests results are cached when they are run in package list mode as long as there are no differences in the package code. This is done in order to avoid repeating tests that are already passing and for which no change has been done.

However, what if we didn't want to use the cache but repeat the tests anyway?

Here is where the flag -count comes to our aid.

$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.019s
$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.015s
$ go test -timeout 1s -run ^Test1Sort$ . -count=1
ok      _/home/mcaci/code/github.com/mcaci/dev-art/bench        0.014s

As indicated in the documentation that can be found by running go help testflag:

To disable test caching, use any test flag or argument other than the cacheable flags. The idiomatic way to disable test caching explicitly is to use -count=1.

And so this was the unexpected usage of the -count flag in go test, a hidden gem in the go documentation. To read more about this and testing in go in general both go help test and go help testflag are great resources to look at.

Hope this was helpful, thanks a lot for your time reading it!

Introduction to benchmarks in Go

Michele Caci — Tue, 26 Jan 2021 17:13:03 +0000

Benchmarks in Go are in many ways similar to unit tests but have key differencies and serve a different purpose. As they are not as known as unit tests in Go, this article aims to give an introductory look at Go's benchmarks: how to create, how to run them, how to read the results and a few pointers to some advanced topics in creating benchmark tests in Go.

Benchmarks are functions that test the performance of Go code, they are included the testing package of the standard Go library and thus available without any dependecies to external libraries.

When executing a benchmark, you are provided with some information about the execution time and, if requested, the memory footprint of the code under benchmark.



$go test -benchmem -run ^$ -bench ^(Benchmark1Sort)$
goos: linux
goarch: amd64
Benchmark1Sort-12          10000        105705 ns/op        8224 B/op          2 allocs/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench 1.083s

Creating a benchmark in Go

To create a benchmark, you need to import in your go file the testing package and create benchmark functions in a similar way that test functions are created.

For example, when defining unit tests, we write a function in the form of func TestAny(t *testing) at the beginning, instead, when we define benchmarks we will create a func BenchmarkAny(b *testing.B).

A notable difference in Go's benchmarks with respect to unit tests is the for loop from 0 to b.N. In fact a benchmark is run multiple times in order to make sure that enough data is collected to improve the accuracy of the performance measurement of the code under benchmark.

The field b.N is not a fixed value but it is dynamically adapted to make sure that the benchmark function is run for at least 1 second.

Here to show is a comparison between a benchmark and a test function:



func Benchmark1Sort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        sort.Ints(generateSlice(1000))
    }
}



func Test1Sort(t *testing.T) {
    slice := generateSlice(1000)
    if len(slice) != 1000 {
        t.Errorf("unexpected slice size: %d", len(slice))
    }
}

Running benchmarks

The starting point for running Go's benchmarks is the go test command and here we will see what we need to make sure we're not just running the unit tests.

Basic usage



go test -bench .

By itself, go test runs only unit tests, so we need to add the flag -bench to instruct go test to run also the benchmarks.

Specifically, this command runs all unit tests and benchmarks in the current package as denoted by the . added as argument for the -bench flag.

The "." value is actually a regular expression which can describe what benchmarks will be executed. For example go test -bench ^Benchmark1Sort$ will run the benchmark named Benchmark1Sort.

Same as when running unit tests, you can add the -v flag for verbose, which will show more details on the benchmarks executed as well as any printed output, logs, fmt.Prints and so on, or add a path (like "./...") to look for benchmarks on a specific package (or all packages and subpackages).



go test -bench . -v
go test -bench . ./...

Running benchmarks only

To filter out all the unit tests from go test's execution the -run ^$ flag should be used.



go test -run ^$ -bench .

The flag -run by itself is used to specify which unit tests should be run. Its argument is a regular expression. When we use ^$ as argument we are effectively filtering out all tests, which means only the benchmarks present in the current package will be executed.

Running multiple times

Simply add the -count parameter to run your benchmark as many times as the specified count: the outcome of all the executions will be shown in the output.



$ go test -bench ^Benchmark1Sort$ -run ^$ -count 4
goos: linux
goarch: amd64
Benchmark1Sort-12          10207            134834 ns/op
Benchmark1Sort-12           7554            175572 ns/op
Benchmark1Sort-12           7904            148960 ns/op
Benchmark1Sort-12           8568            147594 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench     7.339s

This flag is useful when sampling the outcomes of multiple runs to make statistical analisys on the benchmark data.

Reading benchmark results

Let's take again the following example and run it with go test -bench to examine its output.



func Benchmark1Sort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        sort.Ints(generateSlice(1000))
    }
}

With execution time

For the first analysis we run the benchmark with go test -bench ^Benchmark1Sort$ -run ^$



$ go test -bench ^Benchmark1Sort$ -run ^$
goos: linux
goarch: amd64
Benchmark1Sort-12           9252            110547 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench     1.053s

The output shown is present in any benchmark execution and it shows:

The information about the enviroment where Go is run, which is also obtained by running go env GOOS GOARCH (case sensitive)
- In our example they are goos: linux and goarch: amd64.
The benchmark row composed of:
- The name of the benchmark run, Benchmark1Sort-12, that is itself composed of the function name, Benchmark1Sort, followed by the number of CPUs used for the benchmark run, 12.
- The number of times the loop has been executed, 9252.
- The average runtime, expressed in nanoseconds per operation, of the tested function, sort.Ints(generateSlice(1000)), which is in this case 110547 ns/op.
The information about the benchmark overall status, the package(s) under benchmark and the total time for the execution.

Quick note on the number of CPUs: this parameter can be specified by using the -cpu flag; the benchmark will be run multiple times once per CPU defined in the flag.



$ go test -bench ^Benchmark1Sort$ -run ^$ -cpu 1,2,4
goos: linux
goarch: amd64
Benchmark1Sort              9280            113086 ns/op
Benchmark1Sort-2            9379            117156 ns/op
Benchmark1Sort-4            8637            118818 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench     3.234s

If this flag is omitted, a default value is taken from the Go variable GOMAXPROCS and the number of CPUs is not printed in the output when it's equal to 1.

With execution time and memory

To add the information about memory footprint in the output you can add the -benchmem flag as follows.



$ go test -bench ^Benchmark1Sort$ -run ^$ -benchmem

goos: linux
goarch: amd64
Benchmark1Sort-12          10327            116903 ns/op            8224 B/op          2 allocs/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench     2.128s

Two new columns have been added in the output of the benchmark row:

the number of bytes required by the operation under benchmark, 8224 B/op
the number of allocations done by the operation under benchmark, 2 allocs/op

Writing more complex benchmarks

Here are some examples of how to write more complex benchmarks.

StartTimer/StopTimer/ResetTimer

When there is the need to do some setup before actually measuring the time spent to execute code to benchmark, the usage of StartTimer, StopTimer and ResetTimer helps to isolate the bits of code that actually need to be taken into account by the benchmark tools.

Let's take the previous snippet, isolate the creation of the slice from the sorting operation and just measure the execution of the latter.

To do so we can write:



func Benchmark2aSort(b *testing.B) {
    for i := 0; i < b.N; i++ {
        b.StopTimer()
        s := generateSlice(1000)
        b.StartTimer()
        sort.Ints(s)
    }
}

By using b.StopTimer() we signal that from this point on the execution is not going to be part of the benchmark until b.StartTimer() is invoked, which means that in each loop, we only consider the data collected during the execution of sort.Ints(s) for the benchmark.

If we want to prepare the slice at the beginning and make it an invariant for the benchmark we can write instead:



func Benchmark2bSort(b *testing.B) {
    s := generateSlice(1000)
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        sort.Ints(s)
    }
}

By using b.ResetTimer() we discard all data collected so far and start anew the collection of data for the benchmark, effectively ignoring the execution time of the generateSlice call in the overall results.

Benchmark test cases and subbenchmarks

Like tests, also benchmarks can benefit from the structure of test cases and execution loop to create subbenchmarks.

Let's see an example:



func Benchmark3Sort(b *testing.B) {
    benchData := map[string]struct {
        size int
    }{
        "with size 1000":    {size: 1000},
        "with size 10000":   {size: 10000},
        "with size 100000":  {size: 100000},
        "with size 1000000": {size: 1000000},
    }
    b.ResetTimer()
    for benchName, data := range benchData {
        b.StopTimer()
        s := generateSlice(data.size)
        b.StartTimer()
        b.Run(benchName, func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                sort.Ints(s)
            }
        })
    }
}

In this example we make use of a map[string]struct{...} to define our benchmark cases and data as we would do for complex tests with test cases and we invoke the b.Run(name string, f func(*testing.B)) to create subbenchmarks that execute separately our benchmark tests.



$ go test -bench ^Benchmark3Sort$ -run ^$
goos: linux
goarch: amd64
Benchmark3Sort/with_size_1000000-12                   10         130396565 ns/op
Benchmark3Sort/with_size_1000-12                   23210             58078 ns/op
Benchmark3Sort/with_size_10000-12                   1300            865703 ns/op
Benchmark3Sort/with_size_100000-12                   118           8718656 ns/op
PASS
ok      _/home/mcaci/code/github.com/mcaci/dev-art/go-bench     6.670s

Notice that the name of the benchmark cases are appended to the benchmark name as part of the output of the benchmark operation as benchmark_name/benchmark_case_name-number-of-cpus.

Parting thoughts

There is still a long way to describe how benchmarks work in Go and to get deeper knowledge in how to write them effectively. One of the main topics that would need its own article would be the benchmark of concurrent code in Go with the usage of the b.RunParallel calls, however I hope this article helps in giving the basics of benchmarks in Go and some grounds to explore the functionalities and tools that have not mentioned here.

You can find me up on twitter @nikiforos_frees or here on dev.to @mcaci and I'm looking forward to hearing your questions or comments.

This was Michele, thanks for reading!

References

Go's testing package and go cmd testing flags from the Go team
Dave Cheney's How to write benchmarks in Go
Justforfunc's article about benchmarks by Francesc Campoy
Source of benchmark examples inside instana.com's Practical Golang benchmarks
Talk on advanced benchmarks by Daniel Marti given at the DotGo19 conference: Optimizing go code without a blindfold

Clearing your Unix Terminal and other commands using Go scripts

Michele Caci — Wed, 20 Jan 2021 19:12:12 +0000

When you want to run shell commands, like for example clearing the terminal, from a Go program or script this is how you can do:

cmdName := "clear"
cmd := exec.Command(cmdName)
cmd.Stdout = os.Stdout
cmd.Run()

It's important to add the cmd.Stdout = os.Stdout line (you can replace os.Stdout with any io.Writer variable) if you want the output of the command to appear somewhere. This is not necessary however if the command does not output anything.

Launching commands in the terminal is always a useful tool, and very welcome in a Go script. To see it in action, I prepared a fun example of a checkerboard that randomly increases a cell value and prints the equivalent number in Japanese (from 0 to 3).

package main

import (
    "fmt"
    "math/rand"
    "os"
    "os/exec"
    "strings"
    "time"
)

func main() {
    h, l := 8, 8
    board := make([]uint8, h * l)
    for i := 0; i < 120; i++ {
        clear()
        cellFlip(rand.Intn(len(board)), board)
        fmt.Println(plotted(l, board))
        time.Sleep(400 * time.Millisecond)
    }
}

func clear() {
    cmd := exec.Command("clear")
    cmd.Stdout = os.Stdout
    cmd.Run()
}

func cellFlip(i int, b []uint8) {
    if b[i] >= 3 {
        return
    }
    b[i]++
}

func plotted(l int, b []uint8) string {
    var s strings.Builder
    s.Grow(len(b) + l)
    for i, cell := range b {
        s.WriteRune(toRune(cell))
        if i%l != l-1 {
            continue
        }
        s.WriteRune('\n')
    }
    return s.String()
}

func toRune(cell uint8) rune {
    var c rune
    switch cell {
    case 0:
        c = 0x53E3
    case 1:
        c = 0x4E00
    case 2:
        c = 0x4E8C
    case 3:
        c = 0x4E09
    default:
        c = 'x'
    }
    return c
}

0x53E3, 0x4E00, 0x4E8C and 0x4E09 are the codes for 口 (ro which is coming from ゼロ/zero), 一 (ichi) , 二 (ni) and 三 (san) that are the first three numbers in Japanese whose codes are referenced from the Unicode and that can be consulted here.

Please also refer to the `os/exec' package to know more about how to launch commands.

How to create an orphan branch in git

Michele Caci — Mon, 18 Jan 2021 17:37:56 +0000

Sometimes you may need to create a brand new branch on a git project that doesn't contain anything present on the other branches nor shares an history with them, called orphan branch.

This snippet describes how to do it:

BRANCH_NAME=$1
git checkout --orphan $BRANCH_NAME
git rm -rf .
rm .gitignore
echo "# $BRANCH_NAME" > README.md
git add README.md
git commit -m "Initial commit for branch $BRANCH_NAME"
git push origin $BRANCH_NAME

This snippet can be used as a sequence of commands, replacing $1 with any value for the branch name, or in a bash script, for example naming it orphan_branch.sh and running it with sh ./orphan_branch.sh new_branch_name.

More information can be found in the git reference guide at the git checkout documentation.

The meaning of the '.PHONY' target inside a Makefile

Michele Caci — Fri, 08 Jan 2021 17:13:07 +0000

Inside the Makefile of a project you can find a .PHONY element followed by a label and then the same label used as a target for the Makefile.

For example:

.PHONY: build
build:
        go build

.PHONY is actually itself a target for the make commandand and it denotes labels that do not represent files of the project. What does it mean?

In the general case, assuming that there are no files called build in the project, when not using the .PHONY target, like so

build:
        go build

everytime the make build command is invoked it will execute the specified command (in this case go build). However, if there was a file named build, the make build would be bound by the presence of this file and executed only if that file was updated.

The .PHONY target overrides this last behaviour and let's the make build execute no matter of the presence of a file named build.

For a more in-depth overview head to the make command reference.