DEV Community

Cover image for Shell Pipe Wrapper Functions
Ian Pride
Ian Pride

Posted on • Edited on

Shell Pipe Wrapper Functions

#shell #bash #linux #functions

Shell Pipe Wrapper Functions

Create pipe functions from any builtin command, function, or executable program in the Linux shell.

Disclaimer

IntermediateBeginner Friendly

For those who mention xargsthis an alternative method that makes extra logic processing simpler... And I don't like using xargs.

This post assumes that you are familiar with the Linux command line, commands, shell functions, std(in|out), and piping but it should not be too difficult to understand if you are new.

Piping Output In The Linux Command Line

Pipe Efficiency

Piping output is not always the best or fastest way to do things and sometimes can be quite redundant if the command in question has it's own methods of processing output (e.g. find -exec {} \;) as each part of a pipeline is a subshell, but for many things this inefficiency is negligible and sometimes a pipe is almost necessary or, at least, the best way to process output.

Pipe Availability

We all have more than likely used a pipe | in the Linux shell environment to pipe stdout to another builtin command, function, or executable and eventually we realize that not everything can be piped to.

In order to pipe output to a command the command must process stdin (as you would user input) and not all programs do so.

Pipe Wrapper Functions

In this post I will show you how I write a sort of generic shell function for any command, function, or binary file program that doesn't already process stdin and can be used almost completely in place of or with the original command.

This will not be a 100% tutorial, but the examples will have documenting comments and I will explain as I go along.

Pipe Wrapper Logic

The basic logic is as follows:

  1. Test if the file descriptor (FD) is greater than 0 (check if there is stdin already in the piped subshell):
    1. Process the input in whatever way, usually while reading the input passing the input with pass arguments to the command, function, or file.
  2. If the function has not (else) received stdin:
    1. Only pass arguments to the command, function, or file.

Examples

These examples all share the same simple construct/concept while only varying on logic that we need inside the if [ ! -t 0 ]; then block and more often than not, only in the while block.

Most of the time we will only need the basics of what is written here and not extra logic/expression.

function command_pipe {
    if [ ! -t 0 ]; then
        local input
        # input logic here
        # while read or otherwise to create $input
        command "$@" "$input"
    else
        command "$@"
    fi
}
Enter fullscreen mode Exit fullscreen mode

Pipe Wrapper Function Simple Example

Most common usage.

Simple Example STAT Program

This wraps the stat command to be able to accept standard input, especially from the stdout of a piped command at the same time allowing you to still pass normal arguments to the command.

STAT Program Function Wrapper 
function stat_pipe {
    if [ ! -t 0 ]; then # Test if there is input
        local input
        while read -r input; do # read each input 
            if  [[ -f "$input" ]] || # this is extra logic
                [[ -d "$input" ]]; then # more extra logic
                stat "$@" "$input" # process input with command and arguments
            fi
        done
    else stat "$@"; fi # if no input then run the command as usual with arguments
}
Enter fullscreen mode Exit fullscreen mode
STAT Program Function Example

Print my .bash_func files and get the file sizes

 $ printf '%s\n' .bash_func*
.bash_funcs
.bash_funcs_nocomments
.bash_funcs.old
 $ printf '%s\n' .bash_func* | stat_pipe -c %s
90329
53140
59990
 $ stat_pipe -c %s .bash_func*
90329
53140
59990
Enter fullscreen mode Exit fullscreen mode

Pipe Wrapper Function Advanced Example

Something a little more advanced.

Advanced Example RM Program

A wrapper for the rm command that accepts more user input other than the initial input to decide if you want to delete a file or folder or not; if the else block is executed (not from pipe) then it runs the command as normal.

You should, of course, be careful with rm and therefore I added the extra test logic to add extra security.

This example is dependant on Bash as the shell.

RM Program Function Wrapper 
function rm_pipe {
    if [[ ! -t 0 ]]; then
        local input input_user
        while read -r input; do
            if  [[ -f "$input" ]] ||
                [[ -d "$input" ]]; then
                printf 'Would you like to delete: %s: (y/[N])?\n' "$input"
                read -u 1 input_user # '-u 1' read from keyboard rather than stdin; defaults to No.
                if [[ "$input_user" =~ ^([yY]|[yY][eE][sS])$ ]]; then
                    rm "$@" "$input" # if user input is yes then process...
                fi
            fi
        done
    else rm "$@"; fi
}
Enter fullscreen mode Exit fullscreen mode
RM Program Function Example 
 $ cd fake
 $ printf '%s\n' *
a
b
c
d
 $ printf '%s\n' * | rm_pipe -rf
Would you like to delete: a: (y/[N])?
y 
Would you like to delete: b: (y/[N])?

Would you like to delete: c: (y/[N])?
no
Would you like to delete: d: (y/[N])?
yes
 $ printf '%s\n' *
b
c
 $
Enter fullscreen mode Exit fullscreen mode

Conclusion

This makes almost any builtin command, function, or executable file program accessible to pipe in the Linux command line. I make these types of functions for many things and they end up saving lots of time and effort so I don't always have to write extra array creating logic first and/or the function does my logic for me.

Top comments (8)

Collapse
 
darkwiiplayer profile image
π’ŽWii πŸ³οΈβ€βš§οΈ •

I don't like xargs

*builds own xargs*

Is there any specific reason for not just using the out-of-the-box xargs instead though?

Also, xargs has the -p option, which can be used to get some parallelism out of your shellscripts.

Collapse
 
thefluxapex profile image
Ian Pride •

Because of other comments about xargs everywhere I'll be doing a post soon explaining why xargs is almost always a last resort.

Collapse
 
taikedz profile image
TaiKedz • • Edited

From what I can see here, this is what the xargs command is for..

cmd1 | xargs cmd2 will run cmd1, its output is then piped through and xargs will take care of running cmd2 on every line of piped data.

The rm command would be written as

path_gen_cmd | xargs rm -i
Enter fullscreen mode Exit fullscreen mode

which includes the built-in interactive element of prompting the user for each item.

Collapse
 
thefluxapex profile image
Ian Pride • • Edited

But my prototype method allows all kinds of extra logic processing and I don't like using xargs.

Collapse
 
taikedz profile image
TaiKedz • • Edited

If personal preference, fair, but at that point it's no longer "generic" πŸ˜‰

Seizing the input from keyboard whilst within a pipe by using read -u 1 merits a more significant callout though, that's handy to know...!

Thread Thread
 
thefluxapex profile image
Ian Pride • • Edited

Fair enough on the `no longer "generic", but I usually don't add too much extra and tend to add these functions in my hybrid script/function method so it's a bit more portable for me. Certainly preference though, was just providing alternatives. Appreciate the feedback.

Collapse
 
darkwiiplayer profile image
π’ŽWii πŸ³οΈβ€βš§οΈ •

Thanks to your avatar I now have a tool song stuck in my head.

Collapse
 
thefluxapex profile image
Ian Pride •

That's an extrememly good thing, no?