I've been tinkering with a command line tool that searches a source file for URLs and generates a report of their status.
slaterslater
/
gustavo
Print a colourized report of all HTTP urls in a file
Gus received a recent PR to add windows & unix style command line arguments. Once this PR was merged, Gus could accept and do the following:
$ python gus.py -f some.file // checks the URLs in some.file
$ python gus.py -v // shows version info
$ python gus.py -h // shows usage instructions
--file, --version and --help work too.
A great start, but why stop there? I knew I wanted Gus to handle many more arguments, so it no longer made sense to check sys.argv against strings through a series of conditional statements. I decided to import argparse to do this heavy lifting for me.
The first thing I did was change the default output of the program. Instead of automatically producing a Rich Text File, output to the console would be standard. If the user wants a Rich Text File as the output, they can pass Gus
-r or --rtf from the command line.
Next I updated the checker function by swapping the http.client library for requests. I renamed get_status_code() to get_status() since it would be returning the code and description as a dictionary. I figured this would be helpful down the road when determining the output format (console or RTF).
def get_status(url):
try:
conn = requests.head(url, timeout=2.5)
code = conn.status_code
series = str(code)[0]
desc = 'UNKN'
if series == '2':
desc = 'GOOD'
elif series == '4':
desc = 'FAIL'
return {'code':code, 'desc':desc}
except: # all exceptions default to status == 400
return {'code':400, 'desc':'FAIL'}
One thing I want to point out before moving on... Notice how all exceptions default to status == 400. I'm not really happy with this, so I opened a new issue to improve exception handling. Hopefully in the near future I (or someone else 🤞) will have the time to work on this.
Whichever output format the user chooses, I decided it will be displayed or produced once all URLs have been checked. Since Gus doesn't yet leverage threading, this can take a while. In the event that Gus is taking a long time, I wanted to show the user that everything is ok by showing the progress. After a little research I came up with the following:
print(f'\r Checking URL {list.index(string)} of {len(list)}', end="\r")
By putting the above print statement in my for loop, each iteration over-writes itself and provides a single line progress update to the user.
Once everything worked as expected, I created two new branches to develop two new features:
At this point I was very happy to have argparse on board. Argparse was already sending a string parameter output to the main function. The value of output could either be "std" or "rtf" and adding "json" into the mix was easy peasy.
In order to support --all, --good, and --bad flags, argparse needed to send a list parameter wanted to the main function. By default, the wanted list would be equal to:
['GOOD','FAIL', 'UNKN']
Passing --good or --bad restrict the values of the above list accordingly. In all cases, the values in wanted get checked against the returned dictionary from get_status() and if there is a match, the formatted string is saved for later output.
Since many functions need to know what the user's output choice is, I decided to make it a global variable instead of passing it from function to function. I'm not sure if this is bad form, but the code looks pretty clean and everything worked as intended.
To date, these are all the options for Gus:
| Command line arguments | Description |
|---|---|
| -f FILENAME, --file FILENAME | location of source file to be checked |
| -a, --all | output includes all results (this is default) |
| -g, --good | output includes only [GOOD] results |
| -b, --bad | output includes only [FAIL] results |
| -r, --rtf | output as colourized rich text file |
| -j, --json | output as JSON |
| -h, --help | show information on how to use the tool and exit |
| -v, --version | show program's version number and exit |
The hard part was the merging of two branches. It was supposed to be a recursive three-way merge, but it ended up a dog's breakfast... I'm certain I screwed up the merging, but the code I want is up on Github so I guess I can't complain. I have two more features planned for Gus, so I'll try merging branches again soon.
Top comments (0)