Automating Simple Things with Python is Awesome

Cover photo by Tim Easley on Unsplash.

This is just a quick post about something that happened to me yesterday at work.

So there I was: working hard as a mechanical engineer. You know, measuring stuff. Squinting at things. Taking some notes. Buying various bolts and screws. The usual.

I had just downloaded some 3D model files for a clamp we were going to use in a fixture -- a real one, sorry software testing folks. This clamp, in fact. As the files came in, I gasped in horror. The files all had the wrong extensions! I had clicked the button to download a folder full of .sldprt (individual parts) and .sldasm (assemblies) files, but they came in as .prt.1 and .asm.1 files. What gives!? And to make matters worse, there were a few XML files thrown into the mix -- useless to me!

So I right-clicked and renamed the first file, changing to the appropriate extension. And, as I right-clicked on the second file (of about 30), my inner developer started getting grumpy. "We don't have to stand for this!" he cried, flinging his tiny inner-developer keyboard across his tiny inner-developer office. (My inner developer startup has offices and doors because I'm not a monster.)

Anyways, back to him. "We don't have to stand for this!" he cried. "We're not some mindless, paper-pushing drones! We can program! We could script this in no time!"

"We could," my inner project manager chimed in. (My inner startup is well-staffed.) "But how much time is that really going to save us? We could be done renaming all of these files in like two minutes."

"Alright," the inner developer replied, bottom lip starting to jut stubbornly. Then I'll do it in 120 seconds or less."

He then said, "Crap. We're on a Windows machine without Bash, and I can't quite remember the PowerShell syntax to manipulate strings. Or, at least, not enough to get it done in time. Oh, wait! We know Python! We can do anything we want!"

So, he quickly guided me to do a search for a way to iterate over a directory and rename files. I had to search because I know there are a few ways of doing it. Was it os, or shutil, or pathlib? I think any one of those would have worked. I know there's a bunch of ways to do what I want, but I needed a way that could be accomplished in 120 seconds of combined Googling and implementing.

Aha! Got it!

"That's 30 seconds down," warned my inner project manager. "Get a move on!"

So I opened up a terminal and fired up ipython, my preferred Python REPL. (Yes I have used bpython. Yes it is amazing. I still like ipython.)

import os

for filename in os.listdir("."):
    if filename.endswith(".1"):
        basename = filename[:-6]
        extension = filename[-5:-2]
        os.rename(filename, basename + ".sld" + extension)

My inner dev team high-fived and clinked their cans of La Croix in celebration.

Was it the prettiest, most elegant code I've ever written? No. But did I get it done in less than 120 seconds? Yes. Did it work? Yes. Did I feel like a wizard? You're darn right.

Python is awesome, and no matter what I do with it, it makes me feel happy.

After-Credits Scene

Now that I'm not on the clock and I'm free to take as much time as I like, I've decided I like this solution better:

from pathlib import Path

for file in Path(".").glob("*.1"):
    name, ext, _ = file.name.split(".")  # There were no other .'s in the names
    file.rename(f"{name}.sld{ext}")

---

Originally posted on assert_not magic?

Comments

[Liana Felt (she/her)]

(My inner startup is well-staffed.)

I hope you never face layoffs 😱🤞

[Ryan Palo]

We’ve got a pretty stable business model, as long as I keep bringing in new knowledge and practice. 😃

[Raunak Ramakrishnan]

Here's one way of doing this using the standard Linux utils

source <(find . -type f -name "*.1" | sed -nr "s/(^.*)\.(.*)\.1/mv \1.\2.1 \1.\2/p")

Explanation:

• source <(CMD) executes text output of CMD as a command.
• find . -type f -name "*.1" finds all files in current directory ending with .1
• sed -nr "s/(^.*)\.(.*)\.1/mv \1.\2.1 \1.\2/p" uses a regex to get the basename and extension before .1 and outputs a move command with .1 removed.
• The output of this pipeline is executed by source

We can also use exec option from find itself but I didn't go down that path :)

[Ryan Palo]

Neat! Thanks for sharing 😄 you could also do it in pure Bash with a for loop and string indexing. But it’s not as nifty as your way.

[Raunak Ramakrishnan]

Yes. Another way I found was using awk instead of source as follows:

find . -type f -name "*.1" | sed -nr "s/(^.*)\.(.*)\.1/mv \1.\2.1 \1.\2/p" | awk '{system($0)}'

This makes the eval of strings part of the pipeline.

[Ali Alhajji]

I once faced a similar issue. My boss, who isn't an IT guy at all, wanted to change the extension of many files. He told me to do it (for I was the new hire then). All the files had a common extension. I did not have to write a code to automate it, I just opened cmd and used the command (rename *.zip *.xlsx)

that's it!

As I was reading through your post, this solution kept popping up in my mind. But then I realized that you had different extensions to change.

[Ryan Palo]

Yeah, it would have been a lot easier if I hadn't needed to preserve some amount of the original extension. But definitely a PowerShell one-liner was my first thought.

[Mark Nicol]

Python is such an awesome glue/hack language. I love your storytelling ability. And thanks for introducing me to glob. That's such a useful command to know for the future.

[Ryan Palo]

Thanks! Glad you liked it!

[rhymes]

Ahah nice story 🤣

[Wajahat Karim 🇵🇰]

If it wasn't for your inner developer, this would have been the most boring and useless article ever.
But your inner developer made it way interesting and amazing to read. Hi five to that inner developer.
PS. I am going to put my next article this way. Wish me luck from your inner developer :)

[Ryan Palo]

Um. Thank you, I think?

[Paul Jacobson]

Thanks for sharing this. I love these little scripts that do awesome, little tasks so well.

[Ryan Palo]

Glad you liked it!

[Andrea Pavoni]

Syntax is important for expressiveness. I love both Python and Ruby for their own strengths. In this case && IMHO, Ruby wins with its one-line that mostly reads like plain english.