Ergonomic Pyhon Text Piping Solution for Linux Shell with pypyp and uv

kurukurin — Sun, 05 Jan 2025 07:28:37 +0000

Abstract

This short blog post is an introduction about a linux text piping solution with pypyp and uv, it can easily reuse all your knowledge and packages about python without learning awk. We focus on telling the reader why choosing it instead of how to use it. If you want to learn more about the usage, visit pypyp's homepage and uv's homepage

Why I won't use `awk` ?

When writing linux shell scripts or commands, awk , sed and grep are powerfull tools for working with texts: You can use grep to find something like ls | grep myname, use sed to replace something and use awk as a turing complete programming languages to deal with more sophisticated cases.
grep and sed are fine. They do one thing and do it very well. But awk is not. As we know, awk is a programming language for text, and it takes more time to learn how to use it comparing to grep and sed. That's the problem, awk is a good text processing tool but not a good programming language.
Comparing to Python, Ruby and Perl, awk is not a general purpose programming language, so the 99% usage of awk is only processing texts in linux shell, and the convenience of that is not worth your time and cognitive loading for learning a new programming language, especially when you're not majoring in shell scripting.
So, life is short, why learning another programming language if you can use the one that you have already learnt?

Why I choose pypyp?

pypyp is a solution. It's a simple (less than 800 lines of code) python script than could help you replace awk , sed and grep with a single command pyp, with all your knowledge about python. Here's a quick example.

uname | pyp 'x.lower()'
ls | uvx pypyp 're.match(r"\S+.c",x)'   # use python regex

pypyp solves many simple but important problems about python -c, it reads stdin to lines variable and split lines to x variable, it also print the last expression automaticlly. Meanwhile it imports some comman packages to make python as easy to use as a text processing language for linux shell as perl and awk.

Why I also use uv?

uv is like the cargo or npm for python. Using pypyp with uvx (works like npx or pipx) is really easy especially your need third party packages for pypyp. For example, I want to use numpy with pypyp, I can simply use uvx --with numpy to add numpy package and use pyp to automaticlly import it.

uvx --with numpy pypyp 'numpy.random.randint(100)'

uv also make installing pypyp easier. Once uv is installed, you can dirctly run uvx pypyp and uvx will download and run it for you.

Conclusion

I found that uvx pypyp is a good alternative for awk, it can reuse all your knowledge about python, without adding more burden for you. But we should also notice that it is not a popular solution for now, and it's better not to share your commands or scripts with others for compatibility.