How to Pass Multiple Arguments to a CLI tool with PowerShell: An example with Databricks CLI

Hi, if you ever worked with a CLI tool you probably know that almost all of them comes with a method for initial configuration, that allows you to save your default settings and also your credentials. This is usually called a profile or a context.

However, what if you need to configure this tool on the fly with arguments you are retrieving from an environment or a CI/CD pipeline variable.

There are multiple ways a CLI tool can achieve this. Some cli tools directly accepts arguments so you can use the tool's authentication command in an automation scenario.

> & myclitool login -url "URL" -username "USERNAME" -secret "PASSWORDorSECRET"

This allows us to authenticate the tool in an automation scenario passing all arguements at once with respective flags such as -url or -secret

However, some CLI tools doesn't come with that option and ask your for it's arguments when you call them and expects you to enter them on the fly.

If you know the path, format, filename format of the configuration file your CLI tool will eventually save, then you can bypass this initial configuration process by simply generating the that configuration file the tool expects in the path it expects it, and that's it. Your CLI tool should read the the content of the file and use the values stored in the file.

However, this method relies on a lot of factors, and puts you in a platform dependent state, as Linux and Windows hosts (either full Operating System or a Container) have different filesystem, and user profile paths. But if you script your inputs, you can just mimic a user input in an automation pipeline. I use PowerShell and Bash in combination on Ubuntu based Azure DevOps hosted agents with no issues at all, while I can develop in Windows.

How do we normally configure Databricks CLI manually for our Development machine

Once we installed databricks cli via pip3 install databricks-cli command then we can get into configuring our tool with databricks configure --token

❯ databricks configure --token
Databricks Host (should begin with https://): https://somedatabricksworkspace.url
Token: somedatabricksapitoken

Then CLI will ask us for 2 arguments, a URL for the Databricks Workspace and a secret, one by one.
Once we enter the hostname and press enter, we will be asked for our token, once we type the token and press enter, then Databricks CLI saves the file under ~\.databricks.cfg in the format:

# Example databricks.cfg file
[DEFAULT]
host = https://somedatabricksworkspace.url
token = somedatabricksapitoken

How to achieve same results for CI/CD automation

Bash

The Bash method is most likely known to most bash coder and linux admin, it's called heredoc or here document and used to pass multiple arguments to a bash via stdin input stream. To find out more about it you can read it here

So by making use of it, assume we have can retrieve the endpoint url and the secret values during runtime and can assign them to two variables. Then we can construct the heredoc by having them separated by a newline hence will pass them one by one as input to our cli tool, Databricks CLI.

dbrclidemo.sh:

#!/bin/bash
DatabricksUrl='https://#{AZURE-DATABRICKS-TENANT-URL}#'
dapiToken='#{AZURE-DATABRICKS-TENANT-DAPI-TOKEN}#'

databricks configure --token <<EOF
${DatabricksUrl}
${dapiToken}
EOF
echo -e "\nDatabricks workspace list:"
databricks workspace list

PowerShell

In PowerShell there is something similar, that allows us to define multi-line string inputs called Here-String

So, again assume we can retrieve the values we want during runtime from a secret vault. Then we can construct a Here-String, print it out with Write-Output cmdlet and pipe the output to our external cli command with the & PowerShell call operator.

pwshclidemo.ps1:

#Pass Arguments to Input prompt received from external CLI tool inside script
$DatabricksUrl = 'https://#{AZURE-DATABRICKS-TENANT-URL}#'
$dapiToken ='#{AZURE-DATABRICKS-TENANT-DAPI-TOKEN}#'

$args = @"
$DatabricksUrl

$dapiToken

"@

Write-Output $args | & databricks configure --token 

Write-Output "`nDatabricks workspace list:"
& databricks workspace list

Summary

You can use above methods in Linux and Windows environments to pass multiple arguments to an external tool. I've covered the example of configuring a CLI tool but this scenario may apply to whole bunch of applications I cannot think of. I hope this helped some PowerShell coders needing and looking for a way to workaround the limitations of the external tool they need to work with.