When I started working with gRPC in Node.js, I followed Google's dynamically generated code base to create my servers and clients. And for basic development, this works really well. At least in a mono repository using relative paths to import the proto files. As we move closer to deployment though, a glaring problem has emerged.
A growing collection of proto files - many of which are sharing messages and models - that need to be imported into various gRPC servers and clients. These servers and clients will be hosted in various Kubernetes clusters.
Create an npm package for each service cluster to house the needed proto files.
At first, I thought I would just write a bash script to copy the needed service and message/model files into an npm package and call it a day. However, I know that you can generate actual server and client from the proto files (in some languages), and I want to be ready for that when it comes to Node.js. Therefore, I decided that my npm packages would hold
protoc generated files.
I decided to make a bash script file to handle the actual generation. I figure we can use/modify this file later to use in a more automated process.
- I remove any existing files. For Node.js,
protoccreates two files, both of which end in
_pb.js. So I used a wildcard to select and delete all files ending that way.
rm -f <path to target>/*_pb.js
- Then I actually generate the files:
grpc_tools_node_protoc --js_out=import_style=commonjs,binary:<path to export directory> --grpc_out=generate_package_definition:<path to export directory> <proto file>
Note: I found that the script needed to change directories and run in the directory housing the proto file. I was not able to figure out how to use relative paths to that directory, though I do believe there is a way to do it.
--grpc_out= flag supports two options:
grpc_js: Generates code with
generate_package_definition: Generates code that does not require any gRPC library, and instead generates
PackageDefinitionobjects that can be passed to the
loadPackageDefinitionfunction provided by both the
I tried both flags and found that the
generate_package_definition worked best in both my servers and clients.
My export directory is the npm package I had created for the cluster that needed the proto file. I am now able to use the package - and therefore the files - in both my servers and clients.
I am very pleased with the results. The script files does need to be manually run with any proto file changes, but I can automate that at a future date.