Python for Bash

Andy Lu — Sun, 29 Jul 2018 02:15:37 +0000

This was originally published on my blog.

Do you like bash scripts? Personally, I don't.

So when I need to write bash scripts, I figure out the commands I need, then
glue them together with Python.

It's been a while since I've needed to do this and while I neglected it before,
the subprocess module is the best way to run these commands.

A Quick Intro to Python's `subprocess.py`

Development Environment

If you are following along with me here, you'll want to be using at least python 3.5. Any version before that and you'll have to use a different API in this module to do the things I'll show you.

The Command

The workhorse of this module is the subprocess.Popen class. There are a ton of arguments you can pass this class, but it can be overwhelming- and not to mention overkill- if you're new to this.

Thankfully, there's a function in the subprocess module that we can interface with instead: subprocess.run().

Here's the function signature with some typical arguments passed in. (I pulled this from the Docs)

subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None,
shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None,
text=None, env=None)*)

That looks pretty complicated, but we can actually ignore most of it and still do pretty neat things. Let's look at some examples.

A Basic Example

import subprocess as sp

result = sp.run("pwd")
print(result)

The output:

/this/is/the/path/to/where/my/terminal/was/
CompletedProcess(args="pwd", returncode=0)

The output of this is the path to the directory you ran this script from; exactly what you would expect. Then there's some CompletedProcess object. This is just an object that stores some information about the command that was run. For this guide, I'm ignoring it, but I'll have links at the end where you can read all about it.

But that's it! That's all you need to run some basic bash commands. The only caveat is you'll be lacking some features of a shell.

To overcome this, let's look at the next example.

A Better Example

import subprocess as sp

result = sp.run("ls -lah > someFile.txt", shell=True)
output = sp.run('ls -lah | grep ".txt"', shell=True)

You may have noticed earlier in the function signature that shell=False, but here I set it to True. By doing so, the command I want actually gets run in a shell. That means I have access to redirection and pipes like I've shown.

A note on running things like this: the command you want to execute must be typed exactly the way you would if you were doing it on a shell. If you read through the Documentation, you'll notice there is a way to run commands as by passing in a list of strings, where each string is either the command or a flag or input to the main command.

I found this confusing because if you follow my "Better Example" way, you are never left wondering if you passed in the arguments correctly. On top of that, you are free to use Python to build up a command based on various conditions.

Here's an example of me doing just that.

A "Real World" Example

#!/usr/bin/env python3

###############################################################################
#                                   Imports                                   #
###############################################################################
import subprocess as sp
from datetime import date

###############################################################################
#                                  Functions                                  #
###############################################################################

def getTodaysDate():
  currDate = date.today()
  return f"{currDate.year}-{currDate.month}-{currDate.day}"

def moveToPosts():
  lsprocess = sp.run("ls ./_drafts", shell=True)
  fileList = lsprocess.stdout.decode('utf-8').strip().split("\n")
  hasNewPost = len(fileList)

  if (hasNewPost == 1):
      print("New post detected")

      srcName = "./_drafts/" + fileList[0]
      destName = " ./_posts/" + getTodaysDate() + "-" + fileList[0]

      command = "mv "+ srcName + destName
      sp.run(command, shell=True)

      return [destName, files[0]]

  elif hasNewPost == 0:
      print("Write more!")
  else:
      print("Too many things, not sure what to do")

def runGit(fullPath, fileName):

  commitMsg = "'Add new blog post'"

  c1 = "git add " + fullPath
  c2 = "git commit -m " + commitMsg

  cmds = [c1,c2]

  for cmd in cmds:
    cp = sp.run(cmd, shell=True)

if __name__ == "__main__":
  pathToPost, fileName = moveToPosts()
  runGit(pathToPost, fileName)
  print("Done")

Since this blog is running thanks to Jekyll, I took advantage of the _drafts folder available to me.

For those of you unfamiliar with Jekyll, _drafts is a folder where you can store blog posts that aren't ready to be published yet. Published posts go in _posts.

The filenames in this folder look like: the-title-of-my-post.md. The filenames for published post that sit in the _posts folder have the same name, but with the year-month-day- attached to the front of the draft name.

With this script, I just have to write a post and drop it into _drafts. Then I open a terminal and run this script. First it looks in _drafts and makes an array of the filenames it found. Anything other than just finding one file will stop the script- I'll improve this one day. With that file name and the help of subprocess.run(), the script moves the draft into _posts, gives it the appropriate name, then commits it to git for me.

Wrap Up

I introduced the subprocess.run() function, gave 3 examples of running bash commands with it, and ended with the script that inspired this post in the first place.

I personally don't have too many uses for bash scripts. When I need one though, I'll definitely be writing it in Python and if it suits your needs, you should too.

Explain Managing Projects Like I'm Five

Andy Lu — Thu, 12 Jul 2018 19:59:50 +0000

I was thinking about it today and I realized I don't know what the best practices are when it comes to managing projects. By that I mean, managing versions of Python for Project A and Project B; or whatever tool you use where versions may differ between projects.

There's a 50% chance 'Managing Projects' is the wrong title for this ELI5. I'm just wondering what are the best ways to sandbox projects or if sandboxing projects is a good practice at all in the first place.

How do you keep up with all of your favorite blogs?

Andy Lu — Sun, 24 Jun 2018 05:48:18 +0000

Do you sign up for every mailing list you can? Use a RSS feed? Bookmark each blog and just visit them all? I'm curious to see everyone's strategy!

DEV Community: Andy Lu

Python for Bash

A Quick Intro to Python's `subprocess.py`

Development Environment

The Command

A Basic Example

A Better Example

A "Real World" Example

Wrap Up

Further Reading

Explain Managing Projects Like I'm Five

How do you keep up with all of your favorite blogs?

DEV Community: Andy Lu

Python for Bash

A Quick Intro to Python's subprocess.py

Development Environment

The Command

A Basic Example

A Better Example

A "Real World" Example

Wrap Up

Further Reading

Explain Managing Projects Like I'm Five

How do you keep up with all of your favorite blogs?

A Quick Intro to Python's `subprocess.py`