Build a small command line interface by using a Python function

Cuadros Nicolas — Sat, 25 Jan 2020 20:53:43 +0000

Hi everyone ! I'm a French student in computer science and today I will speak about a module I made to automate the generation of CLI using the inspect and argparse module of Python. It's my first post on DEV.to. For me It's a challenge on multiple points, on one hand it forced me to write in English (I try my best to avoid mistakes) and in the other It make me communicate about my works and (I hope) receive criticsm about it. Also I hope It will give you new ideas based on the inspect module. Now let's dive into the subject !

Context

It’s been a while since I use Python to build small scripts for Deep Learning algorithms and basically parsing some files. Most of the times these scripts end up with some parameters that I have to tweak before running them. In order to avoid to modify the script each time , I use the argparse module to build a small CLI from them. Unfortunately, in most cases, the script that I want to run with parameters is just one main function with some parameters. Here you can see a small example with the prototype of a function that transform a csv file to a json file and how you have to use the argparse module to build your CLI :

import argparse

def csvToJson(infile,outfile:str="out.json",separator:str="\t"):
    """
    Simple function allowing to build a json file based on a csv one
    :param infile: Name of the file to transform
    :param outfile: Name of the output file
    :param separator: Token use by the csv file to separate columns
    """
    return



def main():
    parser = argparse.ArgumentParser(description="Simple function allowing to build a json file based on a csv one")
    parser.add_argument("infile",help="Name of the file to transform")
    parser.add_argument("-o","--outfile",help="Name of the output file",type=str,default="out.json")
    parser.add_argument("-s","--separator",help="Token use by the csv file",type=str,default="\t")
    args = parser.parse_args()
    return csvToJson(args.infile,args.outfile,args.separator)

if __name__ == "__main__":
    main()

If you try to get the help of the CLI you will end up with this :

usage: test.py [-h] [-o OUTFILE] [-s SEPARATOR] infile

Simple function allowing to build a json file based on a csv one

positional arguments:
  infile                Name of the file to transform

optional arguments:
  -h, --help            show this help message and exit
  -o OUTFILE, --outfile OUTFILE
                        Name of the output file
  -s SEPARATOR, --separator SEPARATOR
                        Token use by the csv file

It's a pretty nice formatting of what we specified in our code, but as you can see, you have to declare a second time every parameter of your function, its argument names and types. I found it a little bit painful to do on a daily basis, so I got the idea to automate all this stuff, by retrieving information from the function prototype and build a command arg parser from this information.

The inspect module

In order to automate this, the start point was the inspect module that contains a function named "getfullargspec". This function allows to find most of the information of your functions like :

Number of arguments and theirs names
Their type annotations
And their default value

For example with the above example, this function give use these informations :

specs = getfullargspec(csvToJson)
print(specs.args)#['infile', 'outfile', 'separator']
print(specs.defaults)#('out.json', '\t')
print(specs.annotations)#{'outfile': <class 'str'>, 'separator': <class 'str'>}

So perfect ! We already have everything we need to automatically build our CLI! Except the name of the function, but you can retrieve it by using the .name private attributes of the function. And so from this small function, I build a library that allows you to build your CLI from your function prototype.

The module

The module is named autofunccli and you can get it via pip :

pip install autofunccli

If we go back to our example, we can now simply use the library to build our CLI :

from autofunccli import cmdfunc

#The cmdfunc function can be use as a decorator.
#It will build a FunctionArgParser object.
@cmdfunc
def csvToJson(infile,outfile:str="out.json",separator:str="\t"):
    """
    Simple function allowing to build a json file based on a csv one
    :param infile: Name of the file to transform
    :param outfile: Name of the output file
    :param separator: Token use by the csv file to separate columns
    """
    return

#You can now use the .main method with the __name__ variable to test if it's
#the main call and in this case, parse the command line arguments and launch the function
csvToJson.main(__name__)

If you try to get the help of the CLI you will end up with this :

usage: test.py [-h] [-o OUTFILE] [-s SEPARATOR] infile

Simple function allowing to build a json file based on a csv one

positional arguments:
  infile                Name of the file to transform : <str>

optional arguments:
  -h, --help            show this help message and exit
  -o OUTFILE, --outfile OUTFILE
                        Name of the output file : <str>( )
  -s SEPARATOR, --separator SEPARATOR
                        : <str>(out.json)

[RETURN]

You will end up with almost the same help as above but, with some differences like the type of each argument and their default values are printed. The code is way much easier to read and you don't have to copy paste everything you have already written in your function prototypes.
This module uses the function annotations to know how to convert the strings, for instance if you specify an int in the annotations, the module will call the int function to parse the string.
Of course the module has some limitations, for instance you can only use some default types of Python and you also don't have the same flexibility as the argparse module (like exclusive arguments).

I haven't told everything about autofunccli, you can build a multi-function CLI, use variadic arguments or even enumerations ! So if you want to see more be sure to check out the repository :

CuadrosNicolas / Python-Command-Function

This project aims to build an easy to use package allowing to use a python function directly as a command line script using the argparse module.

I found this module really interesting to build and learned a lot about the inspect module and how you can make some meta-programming stuff in Python. There is a lot to say about this module and how you can automate some parts of Python and how you can conjugate this type of runtime analysis and the meta class of Python.

If you want to know more about the inspect module you can check the official documentation.

I hope you enjoyed this article and this module, if you think we can improve it, tell me in the comments !