DEV Community: Hui`s Journal of Technology

Developer tools matter!

Hui`s Journal of Technology — Mon, 28 Mar 2022 02:46:59 +0000

May you think developer tools are only relevant to the personal behaviors of developers. From my prior experiences, developer tools may impact the teams collaboration, reliability of applications and even business outcome.
In this article, I will talk about why developer tools matters.

Mind pattern of a team

A shared mind pattern of a team is the lubricant of smooth communication. If a team can form an agreement on a code boilerplate generator, developers of the team can foster a shared mind set of how their code will be structured. Standardization of engineering practices is your friend if you want to scale your development team quickly when your business expands. When I started my project, my team and I only focuses on the features delivery to meet business requirements, and we only had 6 team members, and each of us has an our own set of tools and development habits. In the name of polyglot of Microservices, some of services are constructed in varied ways. As our business grows, the code maintenance workload becomes overwhelming and new features delivery were dragged behind because developers have to spend hours to understand services that are developed by his/her peers. Besides the learning cost, when we try to build the CI/CD pipelines, because of varied code structure, multiple pipelines are needed.

Service reliability

Service reliability are often ignored at the beginning. You must often hear saying of "Let's focus on the features first and fix the reliability issue when business is impacted". By saying this, reliability of services are often ignored until something really bad happens. Reliability must be considered at the beginning. Developer tools often have the reliability features included, that means we can use the tool to have the service built in the correct way so that the reliability features are included inherently.

Service security

Security and reliability are often interconnected. A shared framework and development toolset can enable the authentication, authorization, encryption, key management, and etc inherently. By doing this, the security review are much more efficient. Developers can focus on what really matters to the business logic.

A schema first development toolkit for protobuf

Hui`s Journal of Technology — Fri, 25 Mar 2022 03:27:06 +0000

I am recently working on a project called skemaloop, which is a schema-first or schema driven development toolkit that can help you start your API journey.

You can find it here.
https://skemaloop.dev

It is a project that is still in early stage, and I will keep writing articles to introduce the functions of this toolkit and collect feedback from the dev community.

why IDL matters in API development

Hui`s Journal of Technology — Fri, 25 Mar 2022 03:22:07 +0000

We are in the era of API, and our services are exposed in APIs. When there is a new business requirement coming, a context storming happens and the feature will be degined in the principle of DDD and implemented in the form of APIs. A backend developer starts his work by designing an API contract and share it with his/her frontend peers. Frontend developers or from other team design their functions by the API contract. That is to say, we must work on a solid foundation of API contract so that everyone who are in the scope can work independently and start their work in parallel.

This document talks about the the problem of API contact sharing that we face everyday, and why it is neccessary to reconsider how to improve our productivity by changing our way of API design, sharing and development.

RPC v.s. Restful

The Representational State Transfer (REST) style is an abstraction of the architectural elements within a distributed hypermedia system. - Roy Thomas Fielding.

A RESTful API focuses on the resources that an API client can require from the API service. It is resouce based and usually performs CRUD operations on resource endpoints. A typical RESTful API usually looks like:

POST /users/501/messages HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"message": "Hello!"}

In Contrast, RPC API is action based, which only have get and post functions.

POST /SendUserMessage HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"userId": 501, "message": "Hello!"}

In Contrast, RPC API is action based, which only have *get* and *post* functions.

Use RESTful to replace RPC?

Let`s examine the following example to see if it a good solution.

RESTful Approach

Imagine we are developing a library system that needs to handle the status of titles. When the library admin needs to update the status of tiltles, you may provide APIs like below.

POST /titles/123/in
POST /titles/123/out
POST /titles/123/suspended

`shell
PATCH /titles/123 HTTP/1.1
Host: api.example.com
Content-Type: application/json

{"status": "Out"}

There is a problem that you want to update mutilple titles in batch and want to add background logic when you update a specific status. For example, when you want to set a title to state of "In" and notify all the readers that are in the wating list.

RPC Approach

You can design your API with action based RPC Post function.

`
POST /api/titles.transition_state HTTP/1.1
Host: api.example.com
Content-Type: application/json

{
"title": "123",
"status": "Out",
"user": "U123,U234"
}

By examples above, you can clearly see the benefits to design you API contract in a hybrid way. For resources, you can expose RESTful API to allow your API consumers to perform CRUD actions, and for the cases that requires background logic and data manapulation, the RPC approach is a better solution.

Why IDL matters

Now, let`s consider how to define a API contract. IDL (Interface Definition Language) is a must for developers to share the API specification. There are multiple reasons including:

IDL can be used to generate code artifacts.
IDL has no ambiguity.
IDL can be better managed in a VSC system (Github is my favorite).

I recommend protobuf as our IDL to define the API contract for reasons:

Protobuf can be used to generate the OpenAPI specification, whereas the OpenAPI cannot be used reversely.
Protobuf is widely used in the most popular RPC framework gRPC, which is actively developed and maitained by Google and community.

How to debug your Go application in VSCode

Hui`s Journal of Technology — Wed, 23 Mar 2022 02:04:07 +0000

I am using VScode as my primary editor of coding, and I feel very excited by the VScode ecosystem. However, when I first used it to debug go application, as the debug settings is not that straightforward, I spent one hour to learn how to setup the debug process with VScode.

Before you start, it'd better to install the dlv, the go debugger, manually.

brew install dlv

Validate your dlv installation.

dlv version
Delve Debugger
Version: 1.8.2
Build: $Id: dbb493ec14d1e7753504d016b1e1ef1665b75b16

Then open VSCode in your root directory of your go project. Usually, in the root directory, you can find your main.go, go.mod, and go.sum files.

Open your debug settings, which is a launch.json file saved in your code directory of ".vscode".

Add the following contents:

    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch file",
            "type": "go",
            "request": "launch",
            "mode": "debug",
            "program": "main.go",
            "args": ["arg1", "arg2"]
        }

    ]
}

Then you can start your debugging by press "F5" now.

Create docker image on your new MacBook Pro M1 version

Hui`s Journal of Technology — Thu, 17 Mar 2022 14:10:34 +0000

I just changed my major work laptop to the new MacBook Pro M1 Pro version. I really love this new machine as it can work almost 10 hours without charging. However, as a developer and product manager, sometimes I feel annoyed because of the new architecture.

Recently I work on a open source project and I am trying to optimize the developer experience of this project to allow developers use docker image to execute the CLI, which avoids installing lots of dependencies on developers` laptop.

However, I found I need create a separate Dockerfile for my new Apple M1 Mac.

First, the x86_64 base image cannot be used anymore, I have to explicitly use arm64 version base image.

FROM --platform=arm64 ubuntu

Secondly, I have to add lots of IF statements to detect the architecture of the host.

arch=uname -mif [[ "${arch}" == "arm64" ]]; then if [[ "$OSTYPE" == "darwin"* ]]; then //add your logic here fi if [[ "$OSTYPE" == "linux"* ]]; then //add your logic here fi fi

Finally, I have to create arm64 version Dockerfile and build image with this file on my M1 Mac.

docker build -t xxx . -f Dockerfile.arm64

It will be great if you can give me a better solution.

Test gRPC services with gRPCurl

Hui`s Journal of Technology — Wed, 16 Mar 2022 03:33:19 +0000

Test gRPC services with gRPCurl

The gRPC functions can only be called by gRPC clients. My team just started migrating to gRPC stack, and I found gRPCCurl is a very helpful CLI tool to test and debug.

Installation

brew install grpcurl

Usage

Start your gRPC service and check the service metadata. My service is running on port of 10030.

Use "-plaintext" if your service does not use TLS.

grpcurl -plaintext localhost:10030 describe
grpc.health.v1.Health is a service:
service Health {
  rpc Check ( .grpc.health.v1.HealthCheckRequest ) returns ( .grpc.health.v1.HealthCheckResponse );
  rpc Watch ( .grpc.health.v1.HealthCheckRequest ) returns ( stream .grpc.health.v1.HealthCheckResponse );
}
grpc.reflection.v1alpha.ServerReflection is a service:
service ServerReflection {
  rpc ServerReflectionInfo ( stream .grpc.reflection.v1alpha.ServerReflectionRequest ) returns ( stream .grpc.reflection.v1alpha.ServerReflectionResponse );
}
sample_module.sample_package.SayHi is a service:
service SayHi {
  rpc SayHello ( .sample_module.sample_package.HelloRequest ) returns ( .sample_module.sample_package.HelloReply );
}

Call sample_module.sample_package.SayHi service.

grpcurl -d '{"msg": "jimmy"}' -plaintext localhost:10030 sample_module.sample_package.SayHi/SayHello
{
  "msg": "SayHijimmy",
  "code": "1"
}