My first open source project - super simple cli testing tool

Simon Bäumer — Mon, 01 Apr 2019 08:51:00 +0000

Introduction

In this article I will cover my first open source project commander. It tests cli apps based on tests defined in a simple yaml file.

Check it out: https://github.com/SimonBaeumer/commander

The problem?

For another project I needed a tool which had the following abilities:

Use the same tool for different operating systems (mainly osx, linux, windows)
Simple and easy writing of tests
Simple installation (no heavy lib or language installation)
Language independent
pls not another DSL

Especially the heavy installation and language independence were important for me because I wanted to test an app inside different docker containers.

Sadly there wasn't a single tool for it. bats just works for bash, go's icmd and python's expect require you to install a language and be familiar with it.

Why using `commander`?

Easy installation
Generate tests automatically
Fast execution
Platform independent
Just yaml

Installation

Download the binary and give it execution permissions. If you want to call it directly from your terminal add it your path environment variable.

Linux & osx

# linux amd64
curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-linux-amd64 -o commander
chmod +x commander 

# osx amd64
curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-darwin-amd64 -o commander
chmod +x commander

windows amd64

curl -L https://github.com/SimonBaeumer/commander/releases/download/v1.0.1/commander-windows-amd64.exe -o commander.exe

... or download it manually and add it to your path.

Quick start!

# Let commander create your test!
$ .commander add echo hello dev.to
written to /current/working/dir/commander.yaml

# ... written to the default commander.yaml
$ cat commander.yaml
tests:
  echo hello dev.to:
    exit-code: 0
    stdout: hello dev.to

# Executes the generated commander config
$ commander test
Starting test file commander.yaml...

✓ echo hello dev.to

Duration: 0.003s
Count: 1, Failed: 0

A full `commander.yaml` example

As you can see the test suite is easy to understand and write.

tests:
  echo hello world:
    stdout:
      lines:
        1: hello world # assert a specifc line
      contains:
        - hello world # this is the default
      exactly: hello # Match stdout exactly
    # The same can be done for stderr
    exit-code: 0
    config:
      timeout: 5ms # set timeout for test
      dir: /tmp # set working dir
      env:
        SOME_ENV: value

  it should have a title: # You can also define a title to be more clear...
    command: echo title # ... and define your command under test here
    exit-code: 0

Define default test configs

Further you can define a default configuration for all tests. This can be overwritten by the local test configs.

config: # Define the config globally
  timeout: 5ms
  dir: /tmp
  env:
    KEY: VALUE

tests:
  print env and dir:
    command: "echo $KEY $(pwd)"
    stdout: VALUE /tmp
    exit-code: 0

Debbuging tests

Sometimes you need to debug your commands. This can be done with --verbose flag.

# Generate the test
$ commander add echo hello dev.to

# Execute it with logging output
$ commander test --verbose
Starting test file commander.yaml...

2019/03/28 16:56:48 title: 'echo hello dev.to'  ExitCode:  0
2019/03/28 16:56:48 title: 'echo hello dev.to'  Stdout:  hello dev.to
2019/03/28 16:56:48 title: 'echo hello dev.to'  Stderr:  
✓ echo hello dev.to

Duration: 0.002s
Count: 1, Failed: 0

Executing

# Execute another file instead of the default
$ commander test another.yaml

# Filter tests
$ commander test commander.yaml "echo hello"

Generating tests

# Add a test to the default commander.yaml
$ ./commander add echo hello
written to /tmp/commander.yaml

# Write to a given file
$ ./commander add --file=test.yaml echo hello
written to test.yaml

# Write to stdout and file
$ ./commander add --stdout echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello

written to /tmp/commander.yaml

# Only to stdout
$ ./commander add --stdout --no-file echo hello
tests:
  echo hello:
    exit-code: 0
    stdout: hello

Conclusion

With commander you can easy generate your test cases and execute them on different platforms without learning a new language or installing annoying libs and tools.

Contributions are always warmly welcomed!

I hope some of you can use it in their projects :)

Link: https://github.com/SimonBaeumer/commander

Notice: Please give me feedback for my first blog post!

Hi, I'm Simon BÃ¤umer

Simon Bäumer — Mon, 07 Nov 2016 12:24:00 +0000

I have been coding for 7 years (atm I am 23years young), started coding in School and continued it as a trainee for 3 years. Now I am full employed since Jule 2016 as a Web Developer.

I live in Germany, Steinfurt, and work for the eCommerce company shopware.
The most time I develop PHP applications but I am generally interested in any programming language, i.e. Ruby or Java.
Further I love concepts like LoD, TDD, SOLID, DDD and I love to learn more about it.

Nice to meet you!

You can find me on GitHub as @SimonBaeumer

DEV Community: Simon Bäumer