Building a boilerplate for Microservices — Part 1

Building a boilerplate for Microservices — Part 1

Building projects often involves doing a lot of repetitive tasks, and we as developers are often led to the frustration :

“Why can’t I just work on the freaking business logic and forget everything else?”.

Even I have always had the same issue. I had to work on the platform, setup the project structure, install dependencies, setup the entire stack required for things like monitoring, testing, linting, building, documenting, logging, containerizing, etc. and if working as a team, I have to also get others to do the same. That is when I started looking out for projects which could probably help me get started quicker with the entire stack.

While I found boilerplates and projects which could possibly help me in the process, I noticed a few things:

1. Some of them provided either too less or too many options — I just wanted enough to get me started quickly and not too much of sugar on top.
2. Most of them were outdated — They included lot of outdated dependencies, were not actively maintained and also included a lot of security issues when scanned.
3. Some of them did not match the stack I am interested in working with — I always base my preference for the right technical stack on a lot of factors and not all my preferences were met.
4. Some included vendor/framework lock-in — I don’t prefer to have any tight coupling with any specific vendor or framework and many of them tied me to something specific limiting my ability to migrate or innovate in the future.
5. Most of them were not production grade — While all of them worked well in the development environment, it did not give the reliability, scale and performance I was looking for in a production environment.
6. Some followed their own standards — While it is good to innovate, it is better to always stick to well known and accepted standards in the community rather than building your own standards. Lot of projects lacked this.

After all the research, I started working on making my own boilerplate for the micro-services stack based on Node.js for quite some time now and today,

I am happy to announce v1.0 of Node-skeleton (open to any better names :P ).

Node-skeleton  — https://github.com/tvvignesh/node-skeleton

Further in this article, I will be sharing all the decisions I made in the project, why it was made, the roadmap and how to get started off quickly with this.

Node.js as the server side language

I was looking to use a lightweight, performant platform good at I/O with a good ecosystem of tooling around and something which can scale well in production and also take less time to on board developers. Keeping all this in mind, Node.js became my default choice. You can also read my article here where I explain why I absolutely love the Node.js ecosystem.

ExpressJS as the Node.js framework

I have worked on a lot of projects using ExpressJS as the framework for Node and the thing which I love about it is that, it is minimalistic, widely adopted, has a lot of tooling around, stable and many of the other frameworks depend on it or extend the features of express.

Typescript /Javascript as the language

With a lot of Javascript versions around, and JS being a weakly typed functional programming language, it makes sense to have a superset on top which lets me use all the new amazing features from the latest versions of JS acting as a transpiler, add types if and when needed to the code we write, provides a great set of tooling on top with good IDE support, helps in processing code to remove things like comments at build time, integrates with 3rd party tooling well and Typescript was the choice for me and in addition to this, the projects built on Typescript and the community around it has been really great.

Docker for Containers

Docker has sort of become the De-facto standard for containerizing applications with loads of tooling around, a huge community, lot of options when running in the cloud, and a lot of CNCF projects built with Docker support out of the box. This makes it very easy to get started running the projects on containers with Docker while alternatives like Rkt still exist.

In addition to this, we have bundled a sample docker-compose file to get started with multiple containers without much pain during development.

PM2 for running in Host

While containers are great, sometimes the requirement is to run the Node process directly in the host without running it within containers, we need a process manager to manage it, cluster processes, manage logs, do load balancing, manage resources like memory, CPU, etc. and PM2 became my default choice (alternatives like forever exist but has limited tooling support around) since the project was made completely on Node.js

ESLint for Linting

While it was tempting to use TSLint for linting the project, I came to a realization that the project was not as mature as ESLint and also, ESLint provided support for linting Typescript through eslint-typescript plugin which made the job much easier for me. After configuring this, I started looking out for inspiration for the rules to use for linting and I took a lot of inspiration from Airbnb and XoJS while defining the rules.

VSCode

This editor is a beast and has become insanely popular and I am in love with it. I was an Atom user once and I switched to VSCode just to experiment and I have been loving the journey. You may be wondering why talk about VSCode when talking about the stack. The fact is that VSCode becomes part of the boilerplate since the settings configured in my VSCode editor are shipped alongside to use and having those can help in things like auto-indentation, space/tab corrections and to help you adhere to the coding standards to be followed.

Winston

Winston is an amazing library for Logging in Node.js since it allows a lot of Transports, has a good community around, has support for things like Logging level, supports Logrotation through plugins, and also allows you to extend it if needed. So, it made sense for me to use console as the transport in development environment and file as transport in production.

Typedoc

I was looking for a documentation generator for the code written (something which could also use JSDoc) and since I had used Typescript, Typedoc fitted the equation very well and got the job done for me.

OpenAPI/Swagger

I was looking to use the latest Open API Spec V3.0 to document the APIs in the project and I started digging in and noticed that I can generate the YAML/JSON spec files through tools like this and once generated, I can use libraries like swagger-ui-express to expose them via express endpoints and it worked like a charm.

Nodemon

The next job was to speed up the development process by adding instant restart to the stack and Nodemon was the right candidate for the job given its huge popularity, community and load of options to work with and all I had to do is add a json file with the required options to get the job done.

*Testing with * AVA

While it is very important to write your business logic, I have realized that its equally important to write tests for the code you write and thus, I was looking for a framework which could help me with this and I chose AVA over Mocha and others considering the performance boost it gave, good API, and good standards without globals, etc.

Exposing Metrics to Prometheus

While it is important to work on the business logic of your application, its also very important to monitor its health and state, add alerts if something goes wrong and have a way to track the change in various trends over time. And this is where Prometheus comes in. I have bundled prom-client to help expose metrics to prometheus and it works like a charm.

In the Roadmap

This project undergoes rapid iterations (you may get something new every week) and this is what I have in plan for the near future. Open to suggestions.

1. Add sample Auth Strategies using Passport
2. Add sample Jenkinsfile for CI/CD
3. Add sample implementations for Kafka producers & consumers
4. Add sample Kubernetes configs
5. Add support for envoy and istio
6. Add support for GraphQL

and a lot more on the way. Star/Watch the project to follow it closely. Will be back with some more news shortly.

Getting Started with node-skeleton

Please refere the README of the project: https://github.com/tvvignesh/node-skeleton for instructions on how you can quickly get started within 5 minutes with your project.

Contributing to this project

Contributors are welcome. Feel free to raise pull requests with issues. I would be happy to accept after a proper discussion and review. If you want to financially support, donate to charity on behalf of this project and share the details — I will add you to the donor list.

Using this project

The project is made available open source under the MIT licence and you can use it without asking for any personal or commercial purposes. Do let me know if you do use it so that I can add you to the users list.

If you have any suggestions on what you would like/not like to see in a boilerplate for Microservices, do let me know and I will see how I can add that to the Roadmap. Feel free to use it and let me know how it behaves for you.

I will be back with another blog post before I launch the next major version of the project. Stay Tuned!

---

T.v. Vignesh

Find me on:

Twitter | LinkedIn | Facebook | Github | Medium

Or join our Timecampus Community on:

Twitter | LinkedIn | Facebook | Github | Medium