Golang, Microservices, and Monorepo

I've been coding in golang for the last 2 years (2018 - 2020), building microservices and committing it to a monorepo.
These years have got me thinking on how to structure Go project based on those 2 criteria: microservice and monorepo

Why Microservice

Business comes and go quickly, especially at the early phase of a startup.
The benefits of using microservice are:

1. Each microservice is treated as 1 business domain
2. Easier to spawn a new microservice without having any dependency to other services
3. It's also easier to take down a microservice in case the business initiative failed, without having to refactor any code

But there are also some downsides on using microservice architecture.
In my experience, code sharing is one of the issues. Which can be tackled by using monorepo

Why Monorepo

One benefit of monorepo is code sharing.
But what the f is code sharing???

Imagine a scenario without monorepo:

• There's an e-commerce named TOKOPA'EDI
• They have 2 microservice named payment and invoice
• Everytime a payment is made, it needs to copy the payment information into invoice

When invoice service accept payment information, invoice service do not have type Payment struct in its repository.
Therefore invoice service have to do some duplication of payment struct.

With monorepo, payment struct can be shared across multiple microservices

Project Structure

project
├── cmd                     # contains microservices 
│   ├── '{microservice a}'  # "microservice a" project which contains "main" function to start app
│   │   └── main.go
│   └── '{microservice b}'  # "microservice b" project, also only contains "main" function to start app
│       └── main.go
├── docker                  # dockefile for each microservices
│   ├── '{microservice a}'
│   └── '{microservice b}'
├── internal                # internal package of each microservice. codes inside internal package only visible to its own microservice and cannot be shared
│   ├── '{microservice a}'
│   │   ├── rest            # HTTP entry point for REST API
│   │   ├── grpc            # gRPC entry
│   │   ├── repo            # repository / connector to database
│   │   └── service         # business logic layer which is handled by the microservice
│   └── '{microservice b}'
│       ├── rest
│       ├── grpc
│       ├── repo
│       └── service
├── pkg                     # public package directory which can be shared across project / microservices
│   └── models              # data model which usually 1:1 relation with database
│       ├── something.go
│       └── another.go
└── readme.md

With above project structure, every object / code inside pkg is fair game and can be shared across microservices without having to copy-pasting object from another repository.

CMIIW ✌️

Comments

[James Eastham]

Hi Robin, great introductory write up. I'm going to raise a counter point to the mono repo and specifically your point on sharing the payment struct.

In my experience, the biggest benefit of microservice comes from the decoupling of code. Sharing a struct between two services makes them really tightly coupled.

I've always preferred having separate data models, and using unit/integration tests to make sure the contract isn't broken.

[Robin]

Hi James, thanks for commenting.

I am well aware of the coupling/de-coupling by sharing data model.
Perhaps sharing struct is not the perfect example for this.

Another sharable code is about role & permission checking.
In my workplace we implemented an RBAC system with each microservice / business domain having it's own rule set.
The RBAC engine is shared across microservices in a single package.

[James Eastham]

Hi Robin, thanks for taking the time to reply.

Completely agree with that use case! If the same permission/role checking needs to happen with every service that screams our for shared code.

I think I'd still use an external reference (NuGet in the case of .NET) rather than direct refs to the local source files. But I do see your point now.

Thankyou for clarifying :-)

[Lucas Pelegrino]

The only problem I have with this, and that perhaps there's a solution, is automatic deployment.

If you change just one micro-service, both will be rolled out a update that way, making both services unavailable and destroying one of the points of having micro-service architecture: independent deployment across micro-services.

I much prefer monorepo, but that's a problem I didn't find a solution yet.

[Robin]

Well CICD will always be a problem with a monorepo.
I previously use file change detection to check which services got changes but it's not a perfect solution.

[Lucas Pelegrino]

Yeah, Uber solved the issue I mentioned with a custom tool. They said they were going to open source it, but that was a couple years ago....

[Rodrigo Melo]

@lucaswxp @bastianrob I started a Monorepo with some scripts that check for changes in my well defined structure and it works well for me at this moment. It's not efficient as Bazel is but we're a health tech startup with small codebase, this year we will migrate to use Bazel which is used by Uber's team to improve our build time and do this job of check deps changes.

[fernandosolivas]

Hi Robin, nice write about monorepo.

As James I am to counter the benefits of monorepo. This problem that you listen up, share struct between microservices could be easily solved with private repository with payment or invoice models and imported into your microservice. Go modules is here to help you.

As James pointed, microservices is to decouple code not only APIs. Be aware, because coupling can bring a lot of problems already knows in software development community.

[{{7*7}}]

Awesome

[Steeve Lennmark]

Is there a reason why everyone seems to prefer putting code specific to a microservice inside internal/microservice instead of just putting it inside cmd/?