ArangoDB,Golang,Grpc(The holy Trinity of Microservices)
In this two part series i want us to build a simple bike rental micro-services application to demonstrate the integration of gRPC with graphql-client using go and persistence of data on an ArangoDb database. It will be divided into the following parts:
In part one of this series we will develop two APIs based on the gRPC framework, a "Users API", that will represent people who will be renting our bikes. The second API will be a "Bikes API", we will then have these two services maintain independent database instances on an ArangoDB database.
In our second part we will link the above two services to a graphql-client.
Find the code to this project "here".
The Technologies, architectures and FrameWorks will be implementing and using are:
Golang:
Officially known as the Go programming language. Go is a language that has been built ground-up for concurrency. It was conceptualized and built by Google engineers.Its a statically typed,natively compiled, garbage-collected ,concurrent, post-OOP(object oriented language).
In current landscape of distributed cloud computing, a little primer: A distributed system is a collection of independent computers that appears to its users as a single coherent system.Now, Golang particularly shines in this sphere using Concurrency. Concurrency in Go is the ability for functions to run independent of each other. Its concurrency mechanisms make it easy to write programs that get the most out of multi core and networked machines(distributed systems). To accomplish this it uses Goroutines, which are independent units of work that get scheduled and then executed on available logical processor. Check out their ofial [sficial site.
Microservices.
Micro-services architecture allow developers to decompose our apps into relatively small app(services) that are loosely coupled to each other. These small apps can then be built to address particular unique business concerns independently using different software and hardware stacks, the deployment and scaling once online can also be done independently too.
This design structure provides a lot of advantages like flexibility in development,robustness in deployment and operation and modularity in scaling . Dive deeper here.
gRPC(google Remote Procedure Call)
This is an RPC(Remote Procedure Call) framework developed by engineers at google, it allows us to innplement RPC based Application Programming Interfaces(API's) on HTTP/2 in contrast to REST based APIs.
Due to the use of HTTP/2 gRPC is more efficient, HTTP/2 can make multiple requests in parallel on a long-lived connection in a thread safe way. The payloads in gRPC are binary-based making them smaller than JSON based messages.Futhermore,HTTP/2 has built-in header compression.
Check out their site
GrapQL
Graphql is an API querying language that was developed to cure the shortcomings of REST such as the more prevalent over-fetching and under-fetching. It was specifically designed for flexibility and efficiency. It also allows for rapid Product iterations on our front-end. Dive deeper into this subject here.
Protocol buffer
Protobuf are googles platform agnostic, language neutral data serialization method, that allows us to initially describe our data in the form of messages. It then allows us to define a set of operations on the "messages" we just defined in the request/response format. Dive deeper into this subject here.
Code-Generators
We will be using code generators in our construction process to generate some of the boiler plate code of our application. This feature allows us to spend our time sewing together the services' business logic as opposed to low-level API implementation logic that should be automatically taken care off. We will be using Prototool in this first part and gqlgen in our second part.
ArangoDB
This a NOSQL database built for high availability and high scalability, a perfect fit for implementing persistence in microservices. ArangoDB is an open source native multi-model database that supports graph, document and key-value data models allowing users to freely combine all data models in a single query. Dive deeper into this database and its features here.
Lets start ....
First things first, lets make sure all our code generators are in place and functional,
prototool instalation:
Prototool primarily provides a centralized location of mantaining consistency across an applications Protobuf files especially in large applications.
For linux users,
curl -sSL \
https://github.com/uber/prototool/releases/download/v1.8.0/prototool-$(uname -s)-$(uname -m) \
-o /usr/local/bin/prototool && \
chmod +x /usr/local/bin/prototool
or simply download prototool-Linux-x86_64.tar.gz and then extract the binary file placing it in the folder /usr/local/bin to make it executable
Check the version to confirm success of installation:
mykmyk@skynet:~$ prototool version
Version: 1.9.0
Default protoc version: 3.8.0
Go version: go1.12.4
OS/Arch: linux/amd64
Dive deeper into the prototool subject here.
Now lets install protoc...
The full name is protobuf compiler, its used to compile .proto file. Dive deeper here.
For linux users:
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28
$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
update the PATH environment var to aid the compiler locate the above plugin "protoc-gen-go-grpc"
$ export PATH="$PATH:$(go env GOPATH)/bin"
GOPATH must be a root folder for the binary files located in $GOPATH/bin and must be the root folder for projects.
To confirm the success of the above installation, use the following command:
mykmyk@skynet:~$ protoc --version
libprotoc 3.15.8
Project Architecture
The finished project will be made up of three services : two microservices that we will construct in this first part and a graphql gateway service that we will work on in the second part of this series.
The gateway is intended to be the ingress of all the client requests associated with Bikes and their Rentees, and the two services will store their own respective data on ArangoDB. When a Rentee rents a Bike, the Rentee and the Bike are associated with the Bike's ID. Based on this link we can retrieve the Rentees' ID by using the Bike's ID.
Project File Structure
create a root folder "bikerenting" with the subfolders,
- gen to hold our generated Go code,
- Proto to hold our proto files ,
- bikes to hold our implementation of bikes API,
- rentees to hold our implementation of the rentees API and - graph_api to house our graphl-gateway implementaion
- dockerfiles for containerization purposes.
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ ll
total 32
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun 1 23:18 ./
drwxrwxr-x 24 mykmyk mykmyk 4096 Jun 1 23:16 ../
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 bikes/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 db/
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 1 23:18 docker-compose.yml
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 1 23:18 Dockerfile.dev
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 gen/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 graph_api/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 proto/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:17 rentees/
We next want to generate a prototool.yaml config file using the prototool code generation tool that we installed at the inception of this project. To create a prototool.yaml file in the projects root folder lets use the following command:
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ prototool config init
you will notice that a "prototool.yaml" file will be generated,
1 protoc:
2 version: 3.8.0
3 lint:
4 group: uber2
now add the following information;
1 protoc:
2 version: 3.8.0
3 lint:
4 group: uber2
5
6 generate:
7 go_options:
8 import_path: bikerenting
9 plugins:
10 - name: go
11 type: go
12 flags: plugins=grpc
13 output: gen/go
We are simply giving paths to locate/store our generated Go code in the above code.
Create Our Bikes and Rentees API
We are going to generate boiler plate code, i.e client and server stubs for both APIs. What we need to do first is define our data structures and operations on these data structures on our proto files for both APIs.
Lets "cd" into the root proto folder that we had created above and create two new folders to hold our respective proto files, bikes folder and rentees folder, it will look as below.
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto$ ll
total 16
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 1 23:44 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun 1 23:36 ../
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:44 bikes/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:44 rentees/
now move into the "bikes" folder and create the following files;
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto/bikes$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 1 23:46 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 1 23:44 ../
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 1 23:46 bikes_messages.proto
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 1 23:46 bikes.proto
open up the "bikes_messages.proto" and add the following code:
1 syntax = "proto3";
2
3 package bikerenting.grpc.bikes.v1;
4 option go_package = "bikes";
5
6 // Bike definition
7
8 message Bike {
9 string id = 1;
10 string owner_name = 2;
11 string type = 3;
12 string make = 4;
13 string serial = 5;
14 }
Then open up the file "bikes.proto" and add the code below:
syntax = "proto3";
package bikerenting.grpc.bikes.v1;
option go_package = "bikes";
import "proto/bikes/bikes_messages.proto";
//API for managing bikes
service BikesAPI {
//Get all bikes
rpc ListBikes(ListBikesRequest) returns (ListBikesResponse);
//Get bike by id
rpc GetBike(GetBikeRequest) returns (GetBikeResponse);
//Get bikes by ids
rpc GetBikes(GetBikesRequest) returns (GetBikesResponse);
// Get bikes by type
rpc GetBikesByTYPE(GetBikesByTYPERequest) returns (GetBikesByTYPEResponse);
// Get bikes by make
rpc GetBikesByMAKE(GetBikesByMAKERequest) returns (GetBikesByMAKEResponse);
// Get bikes by owner_name
rpc GetBikesByOWNER(GetBikesByOWNERRequest) returns (GetBikesByOWNERResponse);
// Add new bike
rpc AddBike(AddBikeRequest) returns (AddBikeResponse);
// Delete bike
rpc DeleteBike(DeleteBikeRequest) returns (DeleteBikeResponse);
}
message ListBikesRequest {
}
message ListBikesResponse {
repeated Bike bikes = 1;
}
message GetBikeRequest {
string id = 1;
}
message GetBikeResponse {
Bike bike = 1;
}
message GetBikesRequest {
repeated string ids = 1;
}
message GetBikesResponse {
repeated Bike bikes = 1;
}
message GetBikesByTYPERequest {
string type = 1;
}
message GetBikeByTYPEResponse {
repeated Bike bikes = 1;
}
message GetBikesByMAKERequest {
string make = 1;
}
message GetBikesByMAKEResponse {
repeated Bike bikes = 1;
}
message GetBikesByOWNERRequest {
string owner_name = 1;
}
message GetBikesByOWNERResponse {
repeated Bike bikes = 1;
}
message AddBikeRequest {
Bike bike = 1;
}
message AddBikeResponse {
Bike bike = 1;
}
message DeleteBikeRequest {
string id = 1;
}
message DeleteBikeResponse {
}
After defining the two file, one to hold our message structure and another to define operations on our structure.
Next run the commands "go mod init" and "go mod tidy" on your terminal, while still in the bikes folder.
In our code above
;
Proto3; (stands for protocol buffers version 3)specifies what protocol we are using for our API definition, the code generator will then use this version to interpret our defined messages and operations.
option go_package = "bikes"; - In this case we are locally specifying the Go import path in this .proto file.
package package bikerenting.grpc.bikes.v1; - this enable the generator to place generated bikes server-client stub code in this location.
For our rentees, the code will take a similar structure, open the proto/rentees folder and create the following files.
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/proto/rentees$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 00:45 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 1 23:44 ../
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 00:45 rentees_messages.proto
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 00:45 rentees.proto
The file "rentees_messages.proto" will look like:
1 syntax = "proto3";
2 package bikerenting.grpc.rentees.v1;
3 option go_package ="rentees";
4
5 // Customer definition
6 message Rentee {
7 string id = 1;
8 string first_name = 2;
9 string last_name = 3;
10 string National_Id_Number = 4;
11 string phone = 5;
12 string email = 6;
13 repeated string held_bikes = 7;
14 }
The "rentee.proto" file will look as below:
syntax = "proto3";
package tutorial.grpc.rentees.v1;
option go_package = "rentees";
import "proto/rentees/rentees_messages.proto";
//API for managing rentees
service RenteesAPI {
//Get all rentees
rpc ListRentees(ListRenteesRequest) returns (ListRenteesResponse);
// Get rentee by bike id
rpc GetRenteeByBikeId(GetRenteeByBikeIdRequest) returns (GetRenteeByBikeIdResponse);
// Get rentee by bike type
rpc GetRenteesByBikeTYPE(GetRenteesByBikeTYPERequest) returns (GetRenteesByBikeTYPEResponse);
// Get rentee by bike make
rpc GetRenteesByBikeMAKE(GetRenteesByBikeMAKERequest) returns (GetRenteesByBikeMAKEResponse);
// Get rentee by bike owner
rpc GetRenteesByBikeOWNER(GetRenteesByBikeOWNERRequest) returns (GetRenteesByBikeOWNERResponse);
// Get rentee by id
rpc GetRentee(GetRenteeRequest) returns (GetRenteeResponse);
// Add new rentee
rpc AddRentee(AddRenteeRequest) returns (AddRenteeResponse);
// Update rentee
rpc UpdateRentee(UpdateRenteeRequest) returns (UpdateRenteeResponse);
}
message ListRenteesRequest {
}
message ListRenteesResponse {
repeated Rentee rentees = 1;
}
message GetRenteeByBikeIdRequest{
string id = 1;
}
message GetRenteeByBikeIdResponse {
Rentee rentee = 1;
}
message GetRenteesByBikeTYPERequest{
string type = 1;
}
message GetRenteeByBikeTYPEResponse {
repeated Rentee rentees = 1;
}
message GetRenteeByBikeMAKERequest{
string make = 1;
}
message GetRenteeByBikeMAKEResponse {
repeated Rentee rentees = 1;
}
message GetRenteeByBikeOWNERRequest{
string owner_name = 1;
}
message GetRenteeByBikeOWNERResponse {
repeated Rentee rentees = 1;
}
message GetRenteeRequest {
string id = 1 ;
}
message GetRenteeResponse {
Rentee rentee = 1;
}
message AddRenteeRequest {
Rentee rentee = 1;
}
message AddRenteeResponse {
Rentee rentee = 1;
}
message UpdateRenteeRequest {
Rentee rentee = 1;
}
message UpdateRenteeResponse {
Rentee rentee = 1;
}
After defining our rentees and bikes APIs our our .proto files,Run the commands "go mod init" and then run "go mod tidy". we will now genate code for the respective client-server stubs using our prototool:
At our root folder i.e "bikerenting" where our prototool.yaml file is located,lets run the following command:
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting$ prototool generate
Prototool will find all our protofiles and use the prototool.yaml configuration to stub our client-server service interfaces in the path that we had directed those files to be deposited as per our configurations in the prototool.yaml file. Now if you navigate to the folder "gen/go/proto". We find autogenerated support modules , the shell for the API servers and client is ready and thus we can the concentrate of sewing together our business logic or controller (or view is Django) for our two APIs,
The folder "gen/go/proto" looks like :
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto$ ll
total 16
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 2 01:28 ./
drwxr--r-- 3 mykmyk mykmyk 4096 Jun 2 01:28 ../
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 01:28 bikes/
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 01:28 rentees/
Now if we "cd" into bikes folder we find the structure below:
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto/bikes$ ll
total 40
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 01:28 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 2 01:28 ../
-rw-rw-r-- 1 mykmyk mykmyk 2728 Jun 2 01:28 bikes_messages.pb.go
-rw-rw-r-- 1 mykmyk mykmyk 25373 Jun 2 01:28 bikes.pb.go
if we "cd" into the rentees folder, we find the following structure:
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/gen/go/proto/rentees$ ll
total 40
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 01:28 ./
drwxrwxr-x 4 mykmyk mykmyk 4096 Jun 2 01:28 ../
-rw-rw-r-- 1 mykmyk mykmyk 3637 Jun 2 01:28 rentees_messages.pb.go
-rw-rw-r-- 1 mykmyk mykmyk 27424 Jun 2 01:28 rentees.pb.go
You can look inside the file to check the serialization that has been done by grpc to our messages:
Make sure you run the commands , "go mod init" and "go mod tidy", in both the rentees folder and the bikes folder.
Creating database connection.....
Download and install the Arangodb database , the official page here has an easy to follow instruction set on that whole process(Download the community edition).
PS: if you run into the error "
Setting up arangodb3 (3.2.10) ... FATAL ERROR: EXIT_FAILED - "exit with error" dpkg: error processing package arangodb3 (--configure): subprocess installed post-installation script returned error exit status 1 Errors were encountered while processing: arangodb3 E: Sub-process /usr/bin/dpkg returned an error code (1)
check out my answer on this error on stack-overflow here
To confirm if the database is correctly installed , use the command "arangosh" as below:
mykmyk@skynet:~$ arangosh
Please specify a password:
_
__ _ _ __ __ _ _ __ __ _ ___ ___| |__
/ _` | '__/ _` | '_ \ / _` |/ _ \/ __| '_ \
| (_| | | | (_| | | | | (_| | (_) \__ \ | | |
\__,_|_| \__,_|_| |_|\__, |\___/|___/_| |_|
|___/
arangosh (ArangoDB 3.3.0 [linux] 64bit, using jemalloc, VPack 0.1.30, RocksDB 5.6.0, ICU 58.1, V8 5.7.492.77, OpenSSL 1.1.0f 25 May 2017)
Copyright (c) ArangoDB GmbH
Pretty printing values.
Connected to ArangoDB 'http+tcp://127.0.0.1:8529' version: 3.3.0 [server], database: '_system', username: 'root'
Please note that a new minor version '3.7.11' is available
Type 'tutorial' for a tutorial or 'help' to see common examples
127.0.0.1:8529@_system>
create a new database and user on the arangosh shell as follows:
127.0.0.1:8529@_system> db._createDatabase("bikesrentees_db");
true
127.0.0.1:8529@_system> var users = require("@arangodb/users");
127.0.0.1:8529@_system> users.save("root@bikesrentees","rootpassword");
{
"user" : "root@bikesrentees",
"active" : true,
"extra" : {
},
"code" : 201
}
127.0.0.1:8529@_system> users.grantDatabase("root@bikesrentees", "bikesrentees_db");
127.0.0.1:8529@_system>
After creating database and user, keep the credentials and details and from our root folder navigate to the "db" folder, in the db folder that stands for database , create two files , connect.go and query.go,
That folder will now look like :
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/db$ ll
total 8
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 02:06 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun 1 23:36 ../
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 02:06 connect.go
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 02:06 query.go
The connect.go file will contain code that we will invoke when we want to establish a connection to our arangodb database that we just created;
The code will look like below, notice we plug in databasename, databaseuser etc:
package db
import (
"context"
"fmt"
"log"
"os"
"github.com/joho/godotenv"
"github.com/arangodb/go-driver"
"github.com/arangodb/go-driver/http"
)
func parseEnvVars(key string) string {
//load .env file
err := godotenv.Load(".env")
if err != nil {
log.Fatalf("Error loading .env file")
}
return os.Getenv(key)
}
const (
DbHost = "http://127.0.0.1"
DbPort = "8529"
DbUserName = "root@bikesrentees_db"
DbPassword = "rootpassword"
)
type DatabaseConfig struct {
Host string
Port string
Username string
Password string
DatabaseName string
}
func Connect(ctx context.Context, config DatabaseConfig)(db driver.Database, err error){
conn, err := http.NewConnection(http.ConnectionConfig{
Endpoints: []string{fmt.Sprintf("%s:%s", config.Host, config.Port)},
})
if err != nil {
return nil, err
}
cl, err := driver.NewClient(driver.ClientConfig{
Connection: conn,
Authentication: driver.BasicAuthentication(config.Username, config.Password),
})
if err != nil {
return nil, err
}
db, err = cl.Database(ctx, config.DatabaseName)
if driver.IsNotFound(err) {
db, err = cl.CreateDatabase(ctx, config.DatabaseName, nil)
}
return db, err
}
func AttachCollection(ctx context.Context, db driver.Database, colName string)(driver.Collection, error){
col, err := db.Collection(ctx, colName)
if err != nil {
if driver.IsNotFound(err){
col, err = db.CreateCollection(ctx, colName, nil)
}
}
return col, err
}
func GetDbConfig() DatabaseConfig{
dbName := parseEnvVars("ARANGODB_DB")
if dbName == "" {
log.Fatalf("Failed to load environment variable '%s'", "ARANGODB_DB")
}
return DatabaseConfig {
Host: DbHost,
Port: DbPort,
Username: DbUserName,
Password: DbPassword,
DatabaseName: dbName,
}
}
Next open up our query.go file and add the following code
package db
import "fmt"
func ListRecords(collectionName string) string {
const listRecordsQuery = `
FOR record IN %s
RETURN record`
return fmt.Sprintf(listRecordsQuery, collectionName)
}
Remember to run the now monotonous command pair or "go mod init" and "go mod tidy " in the db folder.
Now that we have our database connection and query logic, lets finally create our rentees and bikes servers.
From the root of the project folder head to the bikes folder, create a folder called server and a file main.go, the folder will look like below:
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/bikes$ ll
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun 2 02:19 ./
drwxrwxr-x 8 mykmyk mykmyk 4096 Jun 1 23:36 ../
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 02:19 main.go
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 02:19 server/
"cd" into the just created server folder and create a file called "server.go"
Open the server.go using your favorite text editor and add the following code:
package server
import (
"context"
"fmt"
"log"
"net"
"os"
"github.com/myk4040okothogodo/bikerenting/db"
bikesv1 "github.com/myk4040okothogodo/bikerenting/gen/go/proto/bikes"
"github.com/arangodb/go-driver"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
)
const (
bikesCollectionName = "Bikes"
defaultPort = "60001"
)
type Server struct {
database driver.Database
bikesCollection driver.Collection
}
func NewServer(ctx context.Context, database driver.Database)(*Server, error) {
collection, err := db.AttachCollection(ctx, database, bikesCollectionName)
if err != nil {
return nil, err
}
_, _, err = collection.EnsurePersistentIndex(ctx, []string{"serial"}, &driver.EnsurePersistentIndexOptions{Unique: true})
if err != nil {
return nil, err
}
return &Server{
database: database,
bikesCollection: collection,
}, nil
}
func (s *Server) Run() {
port := os.Getenv("APP_PORT")
if port == "" {
port = defaultPort
}
listener,err := net.Listen("tcp", fmt.Sprintf("0.0.0.0:%s", port))
if err != nil {
log.Fatal("net.Listen failed")
return
}
grpcServer := grpc.NewServer()
bikesv1.RegisterBikesAPIServer(grpcServer, s)
reflection.Register(grpcServer)
log.Printf("Starting Rental Bikes server on port %s", port)
go func() {
grpcServer.Serve(listener)
}()
}
func (s *Server) ListBikes(ctx context.Context, in *bikesv1.ListBikesRequest)(*bikesv1.ListBikesResponse, error){
if in == nil {
return nil, fmt.Errorf("Request is empty")
}
cursor, err := s.database.Query(ctx, db.ListRecords(s.bikesCollection.Name()), nil)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over documents: %s", err)
}
defer cursor.Close()
allBikes := []*bikesv1.Bike{}
for {
bike := new(bikesv1.Bike)
var meta driver.DocumentMeta
meta, err := cursor.ReadDocument(ctx, bike)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
return nil, fmt.Errorf("Failed to read bike document: %s", err)
}
bike.Id = meta.Key
allBikes = append(allBikes, bike)
}
return &bikesv1.ListBikesResponse{Bikes: allBikes}, nil
}
func (s *Server) GetBike(ctx context.Context, in *bikesv1.GetBikeRequest)(*bikesv1.GetBikeResponse, error){
if in == nil || in.Id == "" {
return nil, fmt.Errorf("Bike id is not provided")
}
bike := new(bikesv1.Bike)
meta, err := s.bikesCollection.ReadDocument(ctx, in.Id, bike)
if err != nil {
if driver.IsNotFound(err){
err = fmt.Errorf("Bike with id '%s' not found", in.Id)
} else {
err = fmt.Errorf("Failed to get bike with id '%s'", in.Id, err)
}
return nil, err
}
bike.Id = meta.Key
return &bikesv1.GetBikeResponse{Bike: bike}, nil
}
func (s *Server) GetBikes(ctx context.Context, in *bikesv1.GetBikesRequest) (*bikesv1.GetBikesResponse, error){
if in == nil || len(in.Ids) == 0 {
return nil, fmt.Errorf("Bikes ids are not provided")
}
const queryBikesByIds = `
FOR bike IN %s
FILTER bike._key in @ids
RETURN bike`
query := fmt.Sprintf(queryBikesByIds, bikesCollectionName)
bindVars := map[string]interface{}{"ids":in.Ids}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over bike docments with query '%s': %s", queryBikesByIds, err)
}
defer cursor.Close()
bikes := []*bikesv1.Bike{}
for {
bike := new(bikesv1.Bike)
meta, err := cursor.ReadDocument(ctx, bike)
if driver.IsNoMoreDocuments(err) {
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("Failed to read bike document: %s", err)
}
bike.Id = meta.Key
bikes = append(bikes, bike)
}
return &bikesv1.GetBikesResponse{Bikes: bikes}, nil
}
func (s *Server) GetBikesByTYPE(ctx context.Context, in *bikesv1.GetBikesByTYPERequest)(*bikesv1.GetBikesByTYPEResponse, error){
if in == nil || in.Type == "" {
return nil, fmt.Errorf("Bike type is not provided")
}
const queryBikeByTYPE = `
FOR bike IN %s
FILTER bike.type == @type
RETURN bike`
query := fmt.Sprintf(queryBikeByTYPE, bikesCollectionName)
bindVars := map[string]interface{}{"type": in.Type}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByTYPE, err)
}
defer cursor.Close()
bikes := []*bikesv1.Bike{}
for {
bike := new(bikesv1.Bike)
meta, err := cursor.ReadDocument(ctx, bike)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
bike.Id = meta.Key
bikes = append(bikes, bike)
}
return &bikesv1.GetBikesByTYPEResponse{Bikes: bikes}, nil
}
func (s *Server) GetBikesByOWNER(ctx context.Context, in *bikesv1.GetBikesByOWNERRequest)(*bikesv1.GetBikesByOWNERResponse, error){
if in == nil || in.OwnerName == "" {
return nil, fmt.Errorf("Bike owner is not provided")
}
const queryBikeByOWNER = `
FOR bike IN %s
FILTER bike.owner_name == @ownername
RETURN bike`
query := fmt.Sprintf(queryBikeByOWNER, bikesCollectionName)
bindVars := map[string]interface{}{"owner_name": in.OwnerName}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByOWNER, err)
}
defer cursor.Close()
bikes := []*bikesv1.Bike{}
for {
bike := new(bikesv1.Bike)
meta, err := cursor.ReadDocument(ctx, bike)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
bike.Id = meta.Key
bikes = append(bikes, bike)
}
return &bikesv1.GetBikesByOWNERResponse{Bikes: bikes}, nil
}
func (s *Server) GetBikesByMAKE(ctx context.Context, in *bikesv1.GetBikesByMAKERequest)(*bikesv1.GetBikesByMAKEResponse, error){
if in == nil || in.Make == "" {
return nil, fmt.Errorf("Bike make is not provided")
}
const queryBikeByMAKE = `
FOR bike IN %s
FILTER bike.make == @make
RETURN bike`
query := fmt.Sprintf(queryBikeByMAKE, bikesCollectionName)
bindVars := map[string]interface{}{"make": in.Make}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeByMAKE, err)
}
defer cursor.Close()
bikes := []*bikesv1.Bike{}
for {
bike := new(bikesv1.Bike)
meta, err := cursor.ReadDocument(ctx, bike)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
bike.Id = meta.Key
bikes = append(bikes, bike)
}
return &bikesv1.GetBikesByMAKEResponse{Bikes: bikes}, nil
}
//func (s *Server) GetBikeBySERIAL(ctx context.Context, in *bikesv1.GetBikesSERIALRequest)(*bikesv1.GetBikesBySERIALResponse, error){
// if in == nil || in.Serial == "" {
// return nil, fmt.Errorf("Bike serial is not provided")
// }
// const queryBikeBySERIAL = `
// FOR bike IN %s
// FILTER bike.serial == @serial
// RETURN bike`
// query := fmt.Sprintf(queryBikeBySERIAL, bikesCollectionName)
// bindVars := map[string]interface{}{"serial": in.Serial}
// cursor, err := s.database.Query(ctx, query, bindVars)
// if err != nil {
// return nil, fmt.Errorf("Failed to iterate over bike documents with query '%s': %s", queryBikeBySERIAL, err)
// }
// defer cursor.Close()
// b := new(bikesv1.Bike)
// meta, err := cursor.ReadDocument(ctx, b)
// if driver.IsNoMoreDocuments(err){
// return nil, fmt.Errorf("Bike with SERIAL '%s' not found: %s", in.Serial, err)
// } else if err != nil {
// return nil, fmt.Errorf("Failed to read bike document: %s", err)
// }
// b.Id = meta.Key
// return &bikesv1.GetBikesBySERIALResponse{Bike: b}, nil
//}
func (s *Server) AddBike(ctx context.Context, in *bikesv1.AddBikeRequest)(*bikesv1.AddBikeResponse, error){
if in == nil || in.Bike == nil {
return nil, fmt.Errorf("Bike is not provided")
}
meta, err := s.bikesCollection.CreateDocument(ctx, in.Bike)
if err != nil {
return nil, fmt.Errorf("failed to create bike: %s", err)
}
in.Bike.Id = meta.Key
return &bikesv1.AddBikeResponse{Bike: in.Bike}, nil
}
func (s *Server) DeleteBike(ctx context.Context, in *bikesv1.DeleteBikeRequest)(*bikesv1.DeleteBikeResponse, error) {
if in == nil || in.Id == "" {
return nil, fmt.Errorf("Bike id is not provided")
}
_, err := s.bikesCollection.RemoveDocument(ctx, in.Id)
if err != nil {
return nil, fmt.Errorf("Failed to remove existing bike: %s", err)
}
return &bikesv1.DeleteBikeResponse{}, nil
}
}
Run the commands "go mod init" and "go mod tidy".
All we are doing in the above code is implementing the logic for listing bikes using their ids, makes, types and serial numbers, we also implementing the logic for adding a new back to the system and removing a bike from the system, Now to invoke this server by creating a new server, go back one level "cd .." and open the "main.go" file and add the following code.
1 package main
2
3 import (
4 "context"
5 "log"
6 "os"
7 "os/signal"
8 "syscall"
9 "time"
10 "github.com/myk4040okothogodo/bikerenting/bikes/server"
11 "github.com/myk4040okothogodo/bikerenting/db"
12 )
13
14
15 func main(){
16 ctx, cancelFn := context.WithTimeout(context.Background(), time.Second*5)
17 defer cancelFn()
18
19 database, err := db.Connect(ctx, db.GetDbConfig())
20 if err != nil {
21 log.Fatalf("d.OpenDatabase failed with error: %s", err)
22 }
23
24 srv, err := server.NewServer(ctx, database)
25 if err != nil {
26 log.Fatalf("NewServer failed with error: %s", err)
27 }
28
29 srv.Run()
30
31 sigChan := make(chan os.Signal, 1)
32 signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
33 signal := <-sigChan
34 log.Printf("shutting down bikes server with signal: %s", signal)
35
36 }
in the code above we are connecting with our database as per the configurations we supplied in the file "db/connect.go" then creating a New server as per our definition in our "server.go" and running this server, remember this server will be a Go-routine and so we are using a channel to listen for any server killing signals i.e "SIGTERM", to kill our server
We will replicate the above code to create our rentees API, Now head over to the rentees folder from the projects root folder and add the following code , The file structure will be identical to the one we just created above and so to avoid repetition i am not going to be repeating explanations.
mykmyk@skynet:~/code/src/github.com/myk4040okothogodo/bikerenting/rentees$ ll
total 12
drwxrwxr-x 3 mykmyk mykmyk 4096 Jun 2 10:49 ./
drwxrwxr-x 9 mykmyk mykmyk 4096 Jun 2 02:44 ../
-rw-rw-r-- 1 mykmyk mykmyk 0 Jun 2 10:48 main.go
drwxrwxr-x 2 mykmyk mykmyk 4096 Jun 2 10:49 server/
for our "bikerenting/rentees/server/server.go"
package server
import (
"context"
"fmt"
"log"
"net"
"os"
"github.com/myk4040okothogodo/bikerenting/db"
renteesv1 "github.com/myk4040okothogodo/bikerenting/gen/go/proto/rentees"
"github.com/arangodb/go-driver"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
)
const (
renteesCollectionName = "rentees"
defaultPort = "60002"
)
type Server struct {
database driver.Database
renteesCollection driver.Collection
}
func NewServer(ctx context.Context, database driver.Database)(*Server, error){
collection, err := db.AttachCollection(ctx, database, renteesCollectionName)
if err != nil {
return nil, err
}
return &Server {
database: database,
renteesCollection: collection,
}, nil
}
func (s *Server) Run() {
port := os.Getenv("APP_PORT")
if port == "" {
port = defaultPort
}
listener, err := net.Listen("tcp", fmt.Sprintf("0.0.0.0: %s", port))
if err != nil {
log.Print("net.Listen failed")
return
}
grpcServer := grpc.NewServer()
renteesv1. RegisterRenteesAPIServer(grpcServer, s) // use autogenerated code to register the server
reflection.Register(grpcServer)
log.Printf("Starting Rentees server on port %s", port)
go func() {
grpcServer.Serve(listener)
}()
}
func (s *Server) ListRentees(ctx context.Context, in *renteesv1.ListRenteesRequest)(*renteesv1.ListRenteesResponse, error){
if in == nil {
return nil, fmt.Errorf("Request is empty")
}
cursor, err := s.database.Query(ctx, db.ListRecords(s.renteesCollection.Name()), nil)
if err != nil {
return nil, fmt.Errorf("failed to iterate over documents: %s", err)
}
defer cursor.Close()
allRentees := []*renteesv1.Rentee{}
for {
rentee := new(renteesv1.Rentee)
var meta driver.DocumentMeta
meta, err := cursor.ReadDocument(ctx, rentee)
if driver.IsNoMoreDocuments(err) {
break
} else if err != nil {
return nil, fmt.Errorf("Failed to read rentee document: %s", err)
}
rentee.Id = meta.Key
allRentees = append(allRentees, rentee)
}
return &renteesv1.ListRenteesResponse{Rentees: allRentees}, nil
}
func (s *Server) GetRentee(ctx context.Context, in *renteesv1.GetRenteeRequest)(*renteesv1.GetRenteeResponse, error) {
if in == nil || in.Id == "" {
return nil, fmt.Errorf("Rentee id is not provided")
}
rentee := new(renteesv1.Rentee)
meta, err := s.renteesCollection.ReadDocument(ctx, in.Id, rentee)
if err != nil {
if driver.IsNotFound(err) {
err = fmt.Errorf("Rentee with id '%s' not found", in.Id)
} else {
err = fmt.Errorf("Failed to get rentee with id '%s':'%s'", in.Id, err)
}
return nil, err
}
rentee.Id = meta.Key
return &renteesv1.GetRenteeResponse{Rentee: rentee}, nil
}
func (s *Server) GetRenteeByBikeId(ctx context.Context, in *renteesv1.GetRenteeByBikeIdRequest)(*renteesv1.GetRenteeByBikeIdResponse, error){
if in == nil || in.Id == "" {
return nil, fmt.Errorf("Bike id is not provided")
}
const queryRenteeByBikeId = `
FOR rentee IN %s
FOR bikeId IN rentee.held_bikes
FILTER bikeId == @bikeId
RETURN rentee`
query := fmt.Sprintf(queryRenteeByBikeId, renteesCollectionName)
bindVars := map[string]interface{}{"bikeId": in.Id}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeId, err)
}
defer cursor.Close()
r := new(renteesv1.Rentee)
meta, err := cursor.ReadDocument(ctx, r)
if driver.IsNoMoreDocuments(err){
return nil, fmt.Errorf("Rentee that held bike with id %s not found: %s", in.Id, err)
} else if err != nil {
return nil, fmt.Errorf("Failed to read rentee document: %s", err)
}
r.Id = meta.Key
return &renteesv1.GetRenteeByBikeIdResponse{Rentee: r}, nil
}
func (s *Server) GetRenteesByBikeTYPE(ctx context.Context, in *renteesv1.GetRenteesByBikeTYPERequest)(*renteesv1.GetRenteesByBikeTYPEResponse, error){
if in == nil || in.Type == " " {
return nil, fmt.Errorf("Request is empty")
}
const queryRenteeByBikeTYPE = `
FOR rentee IN %s
FOR bikeType IN rentee.held_bikes
FILTER bikeType == @type
RETURN rentee`
query := fmt.Sprintf(queryRenteeByBikeTYPE, renteesCollectionName)
bindVars := map[string]interface{}{"bikeType": in.Type}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeTYPE, err)
}
defer cursor.Close()
rentees := []*renteesv1.Rentee{}
for {
rentee := new(renteesv1.Rentee)
meta, err := cursor.ReadDocument(ctx, rentee)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
rentee.Id = meta.Key
rentees = append(rentees, rentee)
}
return &renteesv1.GetRenteesByBikeTYPEResponse{Rentees: rentees}, nil
}
func (s *Server) GetRenteesByBikeMAKE(ctx context.Context, in *renteesv1.GetRenteesByBikeMAKERequest)(*renteesv1.GetRenteesByBikeMAKEResponse, error){
if in == nil || in.Make == " " {
return nil, fmt.Errorf("Request is empty")
}
const queryRenteeByBikeMAKE = `
FOR rentee IN %s
FOR bikeMake IN rentee.held_bikes
FILTER bikeMake == @make
RETURN rentee`
query := fmt.Sprintf(queryRenteeByBikeMAKE, renteesCollectionName)
bindVars := map[string]interface{}{"bikeMake": in.Make}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeMAKE, err)
}
defer cursor.Close()
rentees := []*renteesv1.Rentee{}
for {
rentee := new(renteesv1.Rentee)
meta, err := cursor.ReadDocument(ctx, rentee)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
rentee.Id = meta.Key
rentees = append(rentees, rentee)
}
return &renteesv1.GetRenteesByBikeMAKEResponse{Rentees: rentees}, nil
}
func (s *Server) GetRenteesByBikeOWNER(ctx context.Context, in *renteesv1.GetRenteesByBikeOWNERRequest)(*renteesv1.GetRenteesByBikeOWNERResponse, error){
if in == nil || in.OwnerName == " " {
return nil, fmt.Errorf("Request is empty")
}
const queryRenteeByBikeOWNER = `
FOR rentee IN %s
FOR bikeOwner IN rentee.held_bikes
FILTER bikeOwner == @owner
RETURN rentee`
query := fmt.Sprintf(queryRenteeByBikeOWNER, renteesCollectionName)
bindVars := map[string]interface{}{"bikeOwner": in.OwnerName}
cursor, err := s.database.Query(ctx, query, bindVars)
if err != nil {
return nil, fmt.Errorf("Failed to iterate over rentee documents with query '%s': %s", queryRenteeByBikeOWNER, err)
}
defer cursor.Close()
rentees := []*renteesv1.Rentee{}
for {
rentee := new(renteesv1.Rentee)
meta, err := cursor.ReadDocument(ctx, rentee)
if driver.IsNoMoreDocuments(err){
break
} else if err != nil {
log.Print(err)
return nil, fmt.Errorf("failed to read rentees document: %s", err)
}
rentee.Id = meta.Key
rentees = append(rentees, rentee)
}
return &renteesv1.GetRenteesByBikeOWNERResponse{Rentees: rentees}, nil
}
func (s *Server) AddRentee(ctx context.Context, in *renteesv1.AddRenteeRequest) (*renteesv1.AddRenteeResponse, error) {
if in == nil || in.Rentee == nil {
return nil, fmt.Errorf("Rentee is not provided")
}
meta, err := s.renteesCollection.CreateDocument(ctx, in.Rentee)
if err != nil {
return nil, fmt.Errorf("Failed to create rentee: %s", err)
}
in.Rentee.Id = meta.Key
return &renteesv1.AddRenteeResponse{Rentee: in.Rentee}, nil
}
func (s *Server) UpdateRentee(ctx context.Context, in *renteesv1.UpdateRenteeRequest)(*renteesv1.UpdateRenteeResponse, error){
if in == nil || in.Rentee == nil || in.Rentee.Id == "" {
return nil, fmt.Errorf("Existing rentee is provided")
}
_, err := s.renteesCollection.ReplaceDocument(ctx, in.Rentee.Id, in.Rentee)
if err != nil {
return nil, fmt.Errorf("Failed to update with id %s", in.Rentee.Id, err)
}
return &renteesv1.UpdateRenteeResponse{Rentee: in.Rentee}, nil
}
For our "bikerenting/rentees/main.go" we will add the following code to that file.
1 package main
2
3 import (
4 "context"
5 "log"
6 "os"
7 "os/signal"
8 "syscall"
9 "time"
10 "github.com/myk4040okothogodo/bikerenting/db"
11 "github.com/myk4040okothogodo/bikerenting/rentees/server"
12 )
13
14
15 func main() {
16 ctx, cancelFn := context.WithTimeout(context.Background(), time.Second*5)
17 defer cancelFn()
18
19 database, err := db.Connect(ctx, db.GetDbConfig())
20 if err != nil {
21 log.Fatalf("db.OpenDatabase failed with error: %s", err)
22 }
23
24 srv, err := server.NewServer(ctx, database)
25 if err != nil {
26 log.Fatalf("NewServer failed with error: %s", err)
27 }
28
29 srv.Run()
30
31 sigChan := make(chan os.Signal, 1)
32 signal.Notify(sigChan, os.Interrupt, syscall.SIGTERM)
33 signal := <-sigChan
34 log.Printf("shutting down Rentees servers with signal: %s", signal)
35 }
Run the commands "go mod init" and "go mod tidy".
The article has grown too big i will have to create an "interlude" article before the second part, this article will majorly dwell on Writing tests for our API, and also using the grpcurl to test our endpoints. See you then.
Top comments (2)
This is awesome! Well done! I have a feeling you would like WunderGraph 👀
Thanks, will check it out!