This is the first post of a 4 part series. If you would like to view this post in a single page, you can follow along on my blog.
There are plenty of resources for the basics of Rust and for protocol buffers + gRPC, so I don’t want to waste your time with heavy introductions. I want to bring you to action as soon as possible.
If you’re here I’ll make a few assumptions about you.
- You can write code in another language, but you have an interest in Rust
- You have basic familiarity with the command line for simple tasks (like listing files with
ls) - You used web service APIs like REST, GraphQL or gRPC in code you’ve written
- You’ve skimmed through the official protocol buffers (v3) docs at least once
- You are looking for some example code that you can copy/paste and modify
Goals for the post
My goal is to walk through writing a small async Rust CLI application. It will take user input from a client, send it to a remote gRPC server, and return output to the client.
The finished code is available in my rust-examples repo, as cli-grpc-tonic-blocking. But I encourage you to follow along, as I will narrate changes while I make them.
What are we writing?
In this example, I will be writing a remote command-line server/client.
The client will take in a command line command and send it to the server who will execute the command and send back the contents of standard out.
For simplicity sake, this example will wait for the execution to complete on the server side before returning output. In a future post I will demonstrate how to stream output back to a client.
I will show you how to:
- Parse command line user input
- Write protocol buffer message types and service interfaces
- Compile protocol buffers into Rust code
- Implement a gRPC client
- Implement a gRPC server (non-streaming)
- Use basic async/await patterns
Bigger picture goals
This is not just a simple Hello World.
I want to provide an example with a realistic application as a foundation. It has potential to be used for something useful, but keep in mind, this example is just a basic script runner and is not secure.
One could run multiple instances of this server on multiple hosts and use the client to run shell commands on each of them similar to continuous integration tools like jenkins, puppet, or ansible. (Hot take: CI is just fancy shell scripting anyway)
I do not recommend running this code as-is in any important environment. For demonstrative and educational purposes only!
Writing the command line interface
The command line interface is the foundation that will allow us to package our gRPC server and client into the same binary. We’re going to start our new crate with the CLI first.
$ cargo new cli-grpc-tonic-blocking
Created binary (application) `cli-grpc-tonic-blocking` package
$ cd cli-grpc-tonic-blocking
We will use a crate called StructOpt. StructOpt utilizes the Clap crate which is a powerful command line parser. But Clap can be a little complicated to use, so structopt additionally provides a lot of convenient functionality Rust a #[derive] attribute so we don’t have to write as much code.
$ cargo new cli-grpc-tonic-blocking
Created binary (application) `cli-grpc-tonic-blocking` package
$ cd cli-grpc-tonic-blocking
cargo.toml
[package]
name = "cli-grpc-tonic-blocking"
version = "0.1.0"
authors = ["T.J. Telan <t.telan@gmail.com>"]
edition = "2018
[dependencies]
# CLI
structopt = "0.3"
In order to bundle our client and server together, we will want to use our CLI to switch between running as a client or running as a server.
Some UI design for the CLI
Note: While we are in development you can use cargo run -- to run our cli binary, and any arguments after the -- is passed as arguments to our binary
Starting the server
When we start our server, we want to pass in the subcommand server
cargo run -- server
Optional arguments for the server
Most of the time our server will listen to a default address and port, but we want to give the user the option to pick something different.
We will provide the option for the server listening address in a flag --server-addr-listen
Using the client
When the user runs a command from our client, we want to use the subcommand run.
$ cargo run -- run
Required positional arguments for the client
Anything after the subcommand run will be the command we pass to the server to execute. A command has an executable name and optionally also arguments.
$ cargo run -- <executable> [args]
Or to illustrate with how one would use this command w/o cargo if it were named remotecli:
$ remotecli run <executable> [args]
Optional arguments for the client
Just like how our server will have a default listening address and port, our client will assume to connect to the default address. We just want to offer the user the option to connect to a different server.
We will provide the option for the server address in a flag --server-addr
The CLI code so far
I’m going to break down the current main.rs into their structs, enums and functions to describe how StructOpt is utilized.
Skip down to the next section All together if you want to check it out in a single code block.
In parts
ApplicationArguments
// This is the main arguments structure that we'll parse from
#[derive(StructOpt, Debug)]
#[structopt(name = "remotecli")]
struct ApplicationArguments {
#[structopt(flatten)]
pub subcommand: SubCommand,
}
Like the comment says, this will be the main struct that you work with to parse args from the user input.
We use derive(StructOpt) on this struct to let the compiler know to generate the command line parser.
The structopt(name) attribute is reflected in the generated CLI help. Rust will use this name instead of the name of the crate, which again is cli-grpc-tonic-blocking. It is purely cosmetic.
The structopt(flatten) attribute is used on the ApplicationArguments struct field. The result effectively replaces this field with the contents of the SubCommand type, which we’ll get to next.
If we didn’t use flatten, then the user would need to use the CLI like this:
$ remotecli subcommand <subcommand> …
But with the flattening we get a simplified form without the subcommand literal.
$ remotecli <subcommand> ...
The reason for this pattern is to allow grouping of the subcommands into a type that we can pattern match on, which is nice for the developer. But at the same time we keep the CLI hierarchy minimal for the user.
SubCommand
// These are the only valid values for our subcommands
#[derive(Debug, StructOpt)]
pub enum SubCommand {
/// Start the remote command gRPC server
#[structopt(name = "server")]
StartServer(ServerOptions),
/// Send a remote command to the gRPC server
#[structopt(setting = structopt::clap::AppSettings::TrailingVarArg)]
Run(RemoteCommandOptions),
}
We’re working with an enum this time. But again, the most important part is the derive(StructOpt) attribute.
The reason to use an enum is to provide some development comfort. Each field in the enum takes in a struct where additional parsing occurs in the event that the subcommand is chosen. But this pattern enables us to not mix that up within this enum and make the code unfocused, and hard to read.
The second most important detail is to notice the comments with 3 slashes ///.
These are doc-comments, and their placement is intentional. Rust will use these comments in the generated help command. The 2 slash comments are notes just for you, the developer, and are not seen by the user.
For the first subcommand, admittedly I named this field StartServer so I could show off using the structopt(name) attribute.
Without the attribute, the user would experience the subcommand transformed by default into the “kebab-case” form start-command. With the name defined on the StartServer field, we tell Rust that we want the user to use server instead.
(You can configure this behavior with the structopt(rename_all) attribute. I won’t be covering that. Read more about rename_all in the docs)
The second subcommand Run... you’ll have to forgive my hand waving.
Remember that StructOpt is built on top of the Clap crate.
Clap is quite flexible, but I thought it was much harder to use. StructOpt offers the ability to pass configuration to Clap and we’re setting a configuration setting w/ respect to the parsing behavior for only this subcommand.
We want to pass a full command from the client to the server. But we don’t necessarily know how long that command will be and we don’t want the full command to be parsed.
The technical description for this kind of CLI parameter is a “Variable-length Argument” or a VarArg in this case. It is a hint for how to parse the last argument so you don’t need to define an end length -- it just trails off.
We are configuring the Run subcommand to tell Rust that this uses a VarArg. See the Clap docs for more info about this and other AppSettings.
ServerOptions
// These are the options used by the `server` subcommand
#[derive(Debug, StructOpt)]
pub struct ServerOptions {
/// The address of the server that will run commands.
#[structopt(long, default_value = "127.0.0.1:50051")]
pub server_listen_addr: String,
}
Our server subcommand has a single configurable option.
The structopt(long) attribute specifies that this is an option that the user will specify with the double-hyphen pattern with the name of the option, which will be in kebab-case by default. Therefore the user would use this as --server-listen-addr.
structopt(default_value) is hopefully self-explanatory enough. If the user doesn’t override, the default value will be used. The default value type is a string slice &str, but structopt is converting it into a String by default.
RemoteCommandOptions
// These are the options used by the `run` subcommand
#[derive(Debug, StructOpt)]
pub struct RemoteCommandOptions {
/// The address of the server that will run commands.
#[structopt(long = "server", default_value = "http://127.0.0.1:50051")]
pub server_addr: String,
/// The full command and arguments for the server to execute
pub command: Vec<String>,
}
Our run subcommand has 2 possible arguments.
- The first,
server_addris an optionalstructopt(long)argument with a default value that aligns with theserverdefault. - The second
commandis a required positional argument. Notice how there is nostructoptattribute. The resulting vector from the variable-length argument. The parser splits up spaces per word, and provides them in order within the Vec. (Matched quotes are interpreted as a single word in our situation).
main()
fn main() -> Result<(), Box<dyn std::error::Error>> {
let args = ApplicationArguments::from_args();
match args.subcommand {
SubCommand::StartServer(opts) => {
println!("Start the server on: {:?}", opts.server_listen_addr);
}
SubCommand::Run(rc_opts) => {
println!("Run command: '{:?}'", rc_opts.command);
}
}
Ok(())
}
Our main() is short and focused.
Our return type is a Result. We return () when things are good, and returns a boxed trait object that implements the std::error::Error trait as our error (the return trait object is boxed, because we don’t Rust doesn’t know how much space to allocate).
We parse the user input using our StructOpt customized ApplicationArguments struct with from_args(). What’s great is invalid inputs are handled, and so we don’t need to spend any time straying from the happy path.
After the parsing, we need to know what action to take next. We’ll either take a server action, or take a client action.
We pattern match on our SubCommand struct, and destructure the enum’s internal structs for the additional arguments.
We eventually will call out to the respective server or client to pass along the args. However for now we call println!() to display the values.
All together
main.rs
use structopt::StructOpt;
// These are the options used by the `server` subcommand
#[derive(Debug, StructOpt)]
pub struct ServerOptions {
/// The address of the server that will run commands.
#[structopt(long, default_value = "127.0.0.1:50051")]
pub server_listen_addr: String,
}
// These are the options used by the `run` subcommand
#[derive(Debug, StructOpt)]
pub struct RemoteCommandOptions {
/// The address of the server that will run commands.
#[structopt(long = "server", default_value = "http://127.0.0.1:50051")]
pub server_addr: String,
/// The full command and arguments for the server to execute
pub command: Vec<String>,
}
// These are the only valid values for our subcommands
#[derive(Debug, StructOpt)]
pub enum SubCommand {
/// Start the remote command gRPC server
#[structopt(name = "server")]
StartServer(ServerOptions),
/// Send a remote command to the gRPC server
#[structopt(setting = structopt::clap::AppSettings::TrailingVarArg)]
Run(RemoteCommandOptions),
}
// This is the main arguments structure that we'll parse from
#[derive(StructOpt, Debug)]
#[structopt(name = "remotecli")]
struct ApplicationArguments {
#[structopt(flatten)]
pub subcommand: SubCommand,
}
fn main() -> Result<(), Box<dyn std::error::Error>> {
let args = ApplicationArguments::from_args();
match args.subcommand {
SubCommand::StartServer(opts) => {
println!("Start the server on: {:?}", opts.server_listen_addr);
}
SubCommand::Run(rc_opts) => {
println!("Run command: '{:?}'", rc_opts.command);
}
}
Ok(())
}
And that’s what we’ve done so far. This will be the full extent of the command line parsing functionality for this example, but we’ll revisit the main() function later.
If you’re following along, this code works with the cargo.toml provided at the top of this section. Play around using cargo.
For example try the following commands:
cargo run --cargo run -- servercargo run -- server -hcargo run -- runcargo run -- run ls -alcargo run -- run -hcargo run -- blahblahblah
We just covered the bare CLI frontend which will allow us to package the implementation of our server and client into a single binary in future posts.
In the next post, I'll cover the creation of the data schema in the Protocol Buffer (protobuf) format.
As well as how we compile the protobufs into Rust code using the Tonic crate.
I hope you'll follow along!
Top comments (0)