Our ventures into Go all started as an internal experiment at Mobile Jazz. As the company name hints, we develop mobile apps.
After releasing an app into the wild, we soon realised we were missing a tool to check what was actually happening to users and how they were interacting with the app - something that would have been very handy in the case of any issues or bugs being reported.
There were a couple of tools around which claimed to help developers in this area, but none of them quite hit the mark, so we decided to build our own. We started by creating a basic set of scripts that quickly evolved into a fully-fledged tool known today as Bugfender!
As this was initially an experiment, we decided to try out a new trending technology. A love of learning and continual education is a key aspect of Mobile Jazz's core values, so we decided to build it using Go; a relatively new programming language developed by Google. It's a new player in the game and there have been many respected developers saying great things about it.
One year later and the experiment has turned into a startup, we've got an incredible tool that's already helping thousands of developers all over the world. Our servers are processing over 200GB of data everyday incoming from more than 7 million devices. After using Go for a year, we'd like to share some of our thoughts and experiences from taking our small experiment to a production server handling millions and millions of logs.
Go Ecosystem
No one in the company had any previous experience using Go. Bugfender was our first dive into the language.
Learning the basics was pretty straight forward. Our previous experiences with C/C++/Java/Objective-C/PHP enabled us to learn Go quickly and get developing in days. There were, of course, a few new and unusual things to learn, including GOPATH and how to deal with packages, but that was expected.
Within a few days, we realized that even being a simplified language by design, Go was extremely powerful. It was able to do everything a modern programming language should: being able to work with JSON, communicate between servers and even access databases with no problems (and with just a few lines of code at that).
When building a server, you should first decide if you're going to use any third party libraries or frameworks. For Bugfender, we decided to use:
Martini
Martini is a powerful web framework for Go. At the time we started the experiment, it was a great solution and to this day, we haven't experienced any problems with it. However if we were to start this experiment again today, we would choose a different framework as Martini is no longer maintained.
We've further experimented with Iris (our current favorite) and Gin. Gin is the successor to Martini and migrating to this will enable us to reuse our existing code.
In the past year, we've realized that Go's standard libraries are really powerful and that you don't really need to rely on a heavy web framework to build a server. It is better to use high-performance libraries that specialize in specific tasks.
Iris is our current favourite and in the future, we'll re-write our servers to use it instead of Martini/Gin, but it's not a priority right now.
Gorm
Some people are fans of ORM and others are not. We decided to use ORM and more specifically, GORM. Our implementation was for the web frontend only, keeping it optimized with hand-written SQL for the log ingestion API. In the beginning, we were really happy, but as time progresses, we've started to find problems and we are soon going to remove it completely from our code and use a lower level approach using a standard SQL library with sqlx.
One of the main problems with GORM is Go's ecosystem. As a new language, there have been many new versions since we started developing the product. Some changes in these new releases are not backwards compatible and so, to use the newest library versions we are frequently rewriting existing code and checking hacks we may have created to solve version issues.
These two libraries are the main building blocks of almost any web server, so it's important to make a good choice because it can be difficult to change later and will affect your server performance.
Third-Party Services
Another important area to consider when creating a real world product is the availability of libraries, third-party services and tools. Here, Go is still lacking maturity, most companies don't yet provide a Go library, so you may need to rely on libraries written by other people where quality isn't always guaranteed.
For example, there are great libraries for using Redis and ElasticSearch, but libraries for other services such as Mixpanel or Stripe are not so good.
Our recommendation before using Go is to check beforehand if there's a good library available for any specific products you may need.
We've also experienced a lot of problems with Go's package management system. The way it handles versions is far from optimal and over the past year, we have run into various problems getting different versions of the same library between different team members. Recently, however, this problem has been almost solved thanks to the new Go feature that supports vendor packages, alongside the gopkg.in service.
Developer Tools
As Go is a relatively new language, you might find the developer tools available are not so great when compared to other established languages like Java. When we started Bugfender it was really hard to use any IDE, none seemed to support Go. But in this last year, this has improved a lot with the introduction of IntelliJ and Visual Studio Code Go plugins.
Finally looking at other Go tools, the debugger is not-so-great and the profiler is even worse, so debugging your code or trying to optimize it can be hard at times.
Heading for Production
This is definitely one of the best things about Go, if you want to deploy something to production you just need to build your binary and send it to the server, no dependencies, no need to install extra software, you only need to be able to run a binary file in your server.
If you're used to dealing with other languages where you require a package manager or need to be careful with a language interpreter you may use, Go is a pleasure to work with.
We are also really happy with Go's stability as the servers never seem to crash. We faced a problem some time ago sending big amounts of data to Go Routines but since then we've rarely seen any crashes. Note: if you need to send a lot of data to a Go Routine, you'll need to be careful as you can have a heap overflow.
If you're interested in performance, we cannot compare to other languages as we have started with Go from scratch, but given the amount of data we process, we feel it's performance is very good, we definitely wouldn't be able to process the same number of requests using PHP so easily.
Conclusions
Over the year we've had our ups and downs regarding Go. At the beginning we were excited, but after the experiment converted to a real product we started unveiling problems. We've thought several times about a complete rewrite in Java, but here we are, still working with Go, and in this past year the ecosystem has had some great improvements that simplified our work.
If you want to build your product using Go, you can be sure it will work, but you need to be really careful with one thing: the availability of developers to hire. There are only a few senior Go developers in Silicon Valley, and finding one elsewhere can be a very difficult task.
Top comments (9)
Great Article, I totally agree with you that it is very easy to learn go, which is why I think the hiring problem is not a big deal. Maybe Im a bit biased here because im in love with go but from the way I see it, if you already know any of the major languages out there, you should be able to read and understand most of go code and the rest is just a matter of time.
I would like the add that there is a new intellij ide for go: jetbrains.com/go/. Its an early build but its pretty stable though.
I also recommend you to not use the iris framework. Its a very toxic framework maintained by one guy. Read this to find out more: Why you should not use iris for your Go projects
I have no hate for your personally so I don't get what incentive I have to "defame" you and I didn't write that article nor do I have any connection with the guy who did so what would be the point of using a clickbait.
The reason I recommend people to not use the iris framework is because the framework is based on fasthttp which you give no credit to.
You also stealed code from httprouter which you didn't give any credit to and when they mentioned it as an issue, you edited the issue to hide from it: link_to_issue.
You cheat on your benchmarks by enabling cache: link_to_issue
If its just one or two small things then you could just let it go but you have been doing this for a long time and when you get called out for it, you just simply erase the history and make it look like nothing happened.
Dear friend,
I wasn't even sure if I had to reply on this but I am doing this for your own protection, I never reply to these type of deflamations.
Now, it's time to see how a person like you can be fooled by reading other's fake-blog post.
Lie 0: I never stole a thing, this is the biggest lie. Did you read the author's comments on this subject? Did you ever read the license of an open-source project?
Did you ever wondered why Iris' router was faster than httprouter when it had one default router? Oh yes, I took it from another Planet, I can't do it myself because you can't, right?
Lie 1: Iris is not based on fasthttp. It's based on HTTP/2 net/http, but when it was (2 versions ago lol) I gave credits to valyala and valyala to me (read both READMEs please), lazy to search & read?
Lie 2: Iris was never used cache by-default, yes we have app.Cache to wrap a handler for cache but benchmarks are not written with these cache handlers, benchmarks repo is also open-source, lazy to read?
Lie 3: I didn't created the benchmarks,a third-party guy which I never had the chance to met did these benchmarks and you could see it if you just clicked on the image of the benchmark, lazy to click?
Beside the fact that you lie without even feeling sorry, you seem lazy, it's time to change that bad behavior of yours!
So what's the reason behind your comment here? When you even don't know the facts by yourself if no clickbait and excuses to deflame Iris?
Read the repository and its code, it's open-source for the sake, don't be lazy.
Laziness is not a permanent property, you can change it starting from today.
You're not "bad".
We all learn from our mistakes eventually.
In order to fix something you need to understand what's wrong first.
When you'll understand what you did you will be able to fix that with an apology, I'll accept that. It doesn't hurts, believe me.
P.S:
I'm really sorry guys. I am not used to reply on these type of comments and maybe that was my mistake. Sorry my personal tune but you may would do the same in my current position.
Thanks,
kataras
Hi Kataras,
I've seen on the web that you are subject to strong negative publicity for some time now. I also see that Iris is under intense development and well maintained. This is a big PLUS for an open source project, especially if people may want to rely on your software for their business.
Some critics that may have been justified in the past are probably not justified anymore. You may not use FastHttp any more, but I'm confident you used it in the past. This not a critic. It's just to explain that you are probably both right and that no one is lying. But I understand that you have become very defensive.
As I was evaluating Go, I managed to create a web site in a week by using Iris. This was an impressive experience and it definitely convinced me that Go was a real tool to do serious work.
I now see that Iris of today has gone a long way since mid 2016 and what may have been true some months and years ago is not true anymore. So I hope for you that the echoes of old critics fade away and all your efforts to build a great and easy to use web framework will finally pay.
However, the way you handle negative feedback may not have changed enough yet. This is harming your image of a dedicated and awesome developer. Better not respond if you're not sure how to turn the exchange into something positive for you.
I wish you the best.
PS: I'm well aware of the very very hard time Greece people are going true. It's great that you can still live you programming passion and share to the world what you develop.
I generally consider people who consider them PHP/C++/Go/Java/pick any single language -programmers not worth hiring, and people who already know a few languages can learn a new one rather easily if they so wish. Go is definitely still difficult to find experienced people for, but I've introduced a few people to it and they were able to pick it up rather quickly and become productive in it, as was I.
Why you should not use Iris for your Go projects
I'm happy that Bugfender uses Iris too. I'm ,always, available for you, we have a community chat and a support forum, please let me updated of your status, I'll be happy to help you in any way I can!
Thanks Gerasimos, we're currently not using Iris but it's a favourite of ours for a future upgrade. Glad to hear we can reach out to you if we need to :)