DEV Community: Yoshiyuki Kato

grpc-mock: A simple mock gRPC server on Node.js

Yoshiyuki Kato — Mon, 07 Jan 2019 17:27:17 +0000

Hi, everyone! Happy new year🌅

Unfortunately I had a bad cold at the very beginning of this year 😷
It was very boring for me to get a rest for some days, so I was developing a Node.js library grpc-mock, with lying down on my bed.

The grpc-mock enables you to build a gRPC mock server by only passing rules about respond. I think this will be useful when you write a test including gRPC calling 😎

Following is a simple example.



const {createMockServer} = require("grpc-mock");
const mockServer = createMockServer({
  protoPath: "/path/to/greeter.proto",
  packageName: "greeter",
  serviceName: "Greeter",
  rules: [
    { method: "hello", input: { message: "Hi" }, output: { message: "Hello" } }
  ]
});
mockServer.listen("0.0.0.0:50051");



syntax="proto3";

package greeter;

service Greeter {
  rpc Hello (RequestGreet) returns (ResponseGreet) {}
}

message RequestGreet {
  string message = 1;
}

message ResponseGreet {
  string message = 1;
}

A rule is described as an object.



{ method: "hello", input: { message: "Hi" }, output: { message: "Hello" } }

This rule make the gRPC server to respond by { message: "Hello" } when hello method called with { message: "Hi" }.

You can also specify the input pattern by Regular Expression string. Following is a rule makes the server to respond by { message: "Hello" } when hello method is called with any argument:



{ method: "hello", input: ".*", output: { message: "Hello" } }

Stream support

grpc-mock supports both of client/server stream. When using stream, you set an appropriate value to streamType property in a rule. Showing example rules with corresponding rpc definitions bellow.

Only client side stream

The server responds by { message: "I'm fine, thank you" } when it receives a streamed sequence of { message: "Hi" } and { message: "How are you?" } in a call of howAreYou method. Please set "client" to streamType property.



{
  method: "howAreYou",
  streamType: "client",
  stream: [
    { input: { message: "Hi" } },
    { input: { message: "How are you?" } },
  ],
  output: { message: "I'm fine, thank you" }
}



rpc HowAreYou (stream RequestGreet) returns (ResponseGreet) {}

Only server side stream

The server responds by a streamed sequence of { message: "Hi, I'm Sana" } and { message: "Nice to meet you too" } when niceToMeetYou method is called with { message: "Hi. I'm John. Nice to meet you" }. Please set "server" to streamType property.



{
  method: "niceToMeetYou",
  streamType: "server",
  stream: [
    { output: { message: "Hi, I'm Sana" } },
    { output: { message: "Nice to meet you too" } },
  ],
  input: { message: "Hi. I'm John. Nice to meet you" }
}



rpc NiceToMeetYou (RequestGreet) returns (stream ResponseGreet) {}

Mutual(client and server) stream

Mutual stream case might be a bit confusing. The server responds by { message: "Hi there" } when receiving { message: "Hi" }, and responds by { message: "I'm fine, thank you." } when receiving { message: "How are you?" } in a call of chat method. Where, inputs of { message: "Hi" } and { message: "How are you?" } must be sent in the order. Please set "mutual" to streamType property.



{
  method: "chat",
  streamType: "mutual",
  stream: [
    { input: { message: "Hi" }, output: { message: "Hi there" } },
    { input: { message: "How are you?" }, output: { message: "I'm fine, thank you." } },
  ]
}



rpc Chat (stream RequestGreet) returns (stream ResponseGreet) {}

Conclusion

In this post, I introduced grpc-mock that is a Node.js library for building gRPC mock server simply. This library is now in developing, your comment, Issue and PR are welcome 🤗

I'm very happy if the lib helps your coding.

YoshiyukiKato / grpc-mock

A simple mock gRPC server on Node.js

grpc-mock

A simple mock gRPC server on Node.js.

const {createMockServer} = require("grpc-mock");
const mockServer = createMockServer({
  protoPath: "/path/to/greeter.proto",
  packageName: "greeter",
  serviceName: "Greeter",
  rules: [
    { method: "hello", input: { message: "test" }, output: { message: "Hello" } },
    { method: "goodbye", input: ".*", output: { message: "Goodbye" } },
    {
      method: "howAreYou",
      streamType: "client",
      stream: [
        { input: { message: "Hi" } },
        { input: { message: "How are you?" } },
      ],
      output: { message: "I'm fine, thank you" }
    },
    
    {
      method: "niceToMeetYou",
      streamType: "server",
      stream: [
        { output: { message: "Hi, I'm Sana" } }

…

View on GitHub

Tips to use MySQL 8.0 on Kubernetes

Yoshiyuki Kato — Thu, 27 Dec 2018 05:56:54 +0000

MySQL 8.0 has many features updated from 5.x. They are totally awesome, however, some features might make some trouble in upgrading from 5.x to 8.0. Change of default authentication plugin is one of such troublesome features. Until 5.x, the default plugin is mysql_native_password. From 8.0, the default plugin is caching_sha2_password. This difference may cause database login failure because it requires a mysql client to support caching_sha2_password. In concrete, you may get a message as follows:

ERROR 2059 (HY000): Authentication plugin 'caching_sha2_password' cannot be loaded

To solve this error, you have to set a parameter to configure authentication plugin before the server starting up. You can set the parameter by putting a config file including following lines:

[mysqld]
default-authentication-plugin=mysql_native_password

Then, how to set the parameter for mysql running on kubernetes? That is the main theme of this post. To put the config file into kubernetes mysql container, you can use ConfigMap.

The ConfigMap is as follows. As shown in code, you can describe the contents of mysql config directly into yaml of ConfigMap.

# config-map.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  ## name of ConfigMap. This will be referred from volume definition
  name: mysql-config
  labels:
    app: mysql
data:
  ## default_auth is the name of config. This will be referred from volume mount definition
  default_auth: |
    [mysqld]
    default_authentication_plugin= mysql_native_password

To put the default_auth data into k8s container as a config file, define volumes and volumeMounts in a mysql pod configuration.

# pod.yaml
kind: Pod
apiVersion: v1
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  containers:
    - name: mysql
      image: mysql:8.0
      imagePullPolicy: Always
      ports:
        - containerPort: 3306
          name: mysql
      env:
      - name: MYSQL_ROOT_PASSWORD # use only for debugging
        value: "root_password"
      - name: MYSQL_DATABASE
        value: "your_database"
      - name: MYSQL_USER
        value: "your_user"
      - name: MYSQL_PASSWORD
        value: "your_password"
      volumeMounts:
      - name: mysql-config-volume ## specify volume name
        mountPath: /etc/mysql/conf.d/default_auth.cnf ## path to mount file
        subPath: default_auth ## name of config
  volumes:
    - name: mysql-config-volume ## volume name
      configMap:
        name: mysql-config ## name of ConfigMap

OK! Then, let’s create these resources onto your k8s cluster, and check if the configuration of the default auth plugin is applied correctly.

$ kubectl apply -f config-map.yaml
$ kubectl apply -f pod.yaml

Check if a pod running up.

$ kubectl get pods

table
NAME      READY   STATUS    RESTARTS   AGE
mysql   1/1     Running   0          5s

Dive into the pod.

$ kubectl exec -it mysql /bin/bash

Check a value of the plugin setting.

root@mysql:/# mysql -u root -D mysql -proot_password

mysql > SELECT user, plugin FROM user WHERE user="your_user";

And you will get following result with table format.

| your_user | mysql_native_password |

Looks good☺️

It’s time to check if it is able to connect the mysql server from outside of the pod. For making easy to connect, create a Service for mysql. This enables pods in the same namespace to access the mysql server by a host name mysql.

# server.yaml
apiVersion: v1
kind: Service
metadata:
  name: mysql
spec:
  ports:
    - port: 3306
      name: mysql
  clusterIP: None
  selector:
    app: mysql

$ kubectl apply -f service.yaml

To connect the server, create a mysql client pod.

# client-pod.yaml
kind: Pod
apiVersion: v1
metadata:
  name: mysql-client
  labels:
    name: mysql-client
spec:
  containers:
  - name: mysql-client
    image: ellerbrock/alpine-mysql-client
    command: ["sleep", "1000"]

$ kubectl apply -f client-pod.yaml

Confirm if the mysql-client pod running up, and dive into the pod.

$ kubectl exec -it mysql-client /bin/ash

~ $ mysql -h mysql -u your_user -D your_database -pyour_password

MySQL [your_database]>

Complete! 🙌

All source codes shown in this post are published in my Github repository. If you are interested in, feel free to clone and try it out!

YoshiyukiKato / mysql8-k8s-example

example of using mysql8.0 on kubernetes

mysql8-k8s-example

Example codes used in my dev.to post.

usage

create all resources

./create

login to mysql server

$ kubectl exec -it mysql-client /bin/ash
$ mysql -h mysql -u your_user -D your_database -pyour_password

MySQL [your_database]>

delete all resources

$ ./delete

View on GitHub

GRPC on Node.js more simply

Yoshiyuki Kato — Sun, 23 Dec 2018 21:42:47 +0000

When developing GRPC application on Node.js, we can use grpc and @grpc/proto-loader. Though these nice packages provides unified APIs to build GRPC server and client, they are too primitive for me to use directly in my application code. I want more simplified API such as createServer and createClient.

So, I am developing grpc-kit. It is a small wrapper library of grpc and @grpc/proto-loader. Ofcourse it provides createServer and createClient😆

Showing an example bellow.

greeter.proto

syntax="proto3";

package greeter;

service Greeter {
  rpc Hello (RequestGreet) returns (ResponseGreet) {}
  rpc Goodbye (RequestGreet) returns (ResponseGreet) {}
}

message RequestGreet {
  string name = 1;
}

message ResponseGreet {
  string message = 1;
}

server.js

const {createServer} = require("grpc-kit");
const server = createServer();

server.use({
  protoPath: "/path/to/greeter.proto",
  packageName: "greeter",
  serviceName: "Greeter",
  routes: {
    hello: (call, callback) => {
      callback(null, { message: `Hello, ${call.request.name}` });
    },
    goodbye: async (call) => {
      return { message: `Goodbye, ${call.request.name}` };
    }
  }
});

server.listen("0.0.0.0:50051");

createServer returns an instance of wrapper class of grpc.Server. The class provides use method for adding a service to provide. You can declaratively configure the service with a routing map. The routing map consists of pairs of method declared in greeter.proto and handler function (both of sync/async functions are available 😎). After the service configured, it starts server by listen method.

client.js

//client.js
const {createClient} = require("grpc-kit");
const client = createClient({
  protoPath: "/path/to/greeter.proto",
  packageName: "greeter",
  serviceName: "Greeter"
}, "0.0.0.0:50051");

client.hello({ name: "Jack" }, (err, response) => {
  if(err) throw err;
  console.log(response.message);
});

client.goodbye({ name: "John" }, (err, response) => {
  if(err) throw err;
  console.log(response.message);
});

createClient returns an instance of grpc.Client. It calls methods declared in greeter.proto and receives a response in callback.

In detail about API, please check out the repository. The library is in developing now. Any comments and advices are welcome. Thank you!

YoshiyukiKato / grpc-kit

Use grpc more simply on Node.js

grpc-kit

Use grpc more simply on Node.js.

quick start

install

$ npm install grpc @grpc/proto-loader grpc-kit

proto

syntax="proto3";

package greeter;

service Greeter {
  rpc Hello (RequestGreet) returns (ResponseGreet) {}
  rpc Goodbye (RequestGreet) returns (ResponseGreet) {}
}

message RequestGreet {
  string name = 1;
}

message ResponseGreet {
  string message = 1;
}

Server

const {createServer} = require("grpc-kit");
const server = createServer();
server.use({
  protoPath: "/path/to/greeter.proto",
  packageName: "greeter",
  serviceName: "Greeter",
  routes: {
    hello: (call, callback) => {
      callback(null, { message: `Hello, ${call.request.name}` });
    },
    goodbye: async (call) => {
      return { message: `Goodbye, ${call.

…

View on GitHub

How to create an original Concourse Resource

Yoshiyuki Kato — Mon, 17 Dec 2018 09:00:09 +0000

Concourse CI is an awesome CI tool developed by pivotal. It provides simplified scheme to define pipelines and rich visualization of their status. To compose a pipeline on Concourse, you can use existing Concourse Resource to define usual tasks such as git operation and slack notification. And also, it is not difficult for you to provide your original concourse resource.

A concourse resource is a docker image. It requires 3 kinds of scripts: check, in and out. You can create your original concourse resource by implementing these scripts, and putting them in the path of /opt/resources with executable permission.

It is required for each script to output result of its execution by json in appropriate format as follows:

check

A list of version info json.

[
  {
      "ref": "a ref value",
      "key": "value"
  }
]

in

version: A version info json.
metadata: A list of metadata json
- item:
  - name: name of a metadata
  - value: value of a metadata

{ 
    "version": {
        "ref": "a ref value",
        "key": "value"
    },
    "metadata": [
        { 
            "name": "metadata_name",
            "value": "metadata_value"
        }
    ]
}

out

version: A version info json.
metadata: A list of metadata json
- item:
  - name: name of a metadata
  - value: value of a metadata

{ 
    "version": {
        "ref": "a ref value",
        "key": "value"
    },
    "metadata": [
        { 
            "name": "metadata_name",
            "value": "metadata_value"
        }
    ]
}

Create Hello World Resource 🌅

Now let's create a super simple concourse resource, helloworld. A project of this resource consists of 3 shell scripts and a Dockerfile. The structure is:

.
├── Dockerfile
└── assets
    ├── check
    ├── in
    └── out

check

#!/bin/bash
echo '[{ "ref": "none",  "message": "Hello World!" }]'

in

#!/bin/bash
echo '{ "version": { "ref": "none", "message": "Hello World" }, "metadata": [{ "name": "message", "value": "Hello World" }] }'

out

#!/bin/bash
echo '{ "version": { "ref": "none", "message": "Hello World" }, "metadata": [{ "name": "message", "value": "Hello World" }] }'

Dockerfile

FROM alpine:3.7
RUN apk add --update --upgrade --no-cache bash
ADD assets /opt/resource
RUN chmod +x /opt/resource/*
WORKDIR /
ENTRYPOINT ["/bin/bash"]

As you know, it is easy to publish the project as a dockerhub repo. You only have to put the project to your github repo, and set automated build to the repo from dockerhub.

...Published? Cool!
Then, let's create a hello world pipeline on your Concourse.

# save this file as helloworld.yaml
resource_types:
- name: helloworld
  type: docker-image
  source:
    # set repo of your helloworld resource
    repository: {{YOUR_ACCOUNT/HELLOWORLD_RESOURCE}}
    ## or use my helloworld resource
    # repository: yoshiyuki/concourse-helloworld-resource

resources:
- name: helloworld
  type: helloworld

jobs:
- name: Hello World
  plan:
  - get: helloworld
  - put: helloworld

$ fly -t ci set-pipeline helloworld -c helloworld.yaml

Hello World! Enjoy! 😎