How to Fuzzy Search: Finding File Names and Contents using Bash Scripting and Commandline Tools

Richard Glen Domingo — Sun, 30 Jun 2024 16:53:52 +0000

This blog explains a bash script that does a fuzzy search against file names and file contents of all the files in the directory.

Introduction

When searching for specific text (or code), you often rely on your IDE or file manager.
But if you're like me who wants to:

1. Search files fast against different directories
1. Filter filenames, then search for text within the filtered files
1. Control which files be ignored

You might find searching using IDE or the file manager to be slow, inefficient and too limiting. That's why I wrote this script.

TLDR (just give me the bash script)

```bash title=vicontrolp wrap showLineNumbers=false
fd \
-E '.key'\
-E '.crt'\
-E 'lock.yaml'\
-E '.jar'\
-E '*.db'\
| xargs \
-I{} awk \
-e '/^([[]-}{#]|[[:space:]])+$/{next;}{ print "{}:" NR ":" $0}'\
{} 2> /dev/null \
| fzf \
--delimiter : \
--preview="bat --color=always --style=plain --highlight-line={2} {1}" \
--preview-window +{2}-5 \
--bind="enter:execute(nvim {1} +{2})"




I have this hooked up with [`zellij`](https://zellij.dev/) and did a keybind with `⌥ + p`.



```nginx title="~/.config/zellij/config.kdl" showLineNumbers=false
keybinds {
    normal {
        ...
        bind "π" {
          Run "vicontrolp" {
            close_on_exit true;
            in_place true;
          }
        }
    }
}

With this, I can trigger this script and search for files whenever I'm
in the comfort of my terminal.

🤔 Understanding the script

Right! Of course you're not some mediocre developer who just copy pastes stuff.
You want to understand how this works so you can expand your knowledge so then you can write your own
developer tools that will enhance your developer experience!

Prerequisites

In order for the script to work, you need the following:

:orange[fd], this commandline tool is required to be installed on your system. See this link to install.
:orange[xargs], this commandline tool is builtin so you don't need to install this. Just listing this here because we'll explain what this does later.
:orange[fzf], this commandline tool is required to be installed on your system. See this link to install.
:orange[bat], this commandline tool is required to be installed on your system. See this link to install.

:orange[creating an executable script], this consists of (1) creating the file, (2) making it executable and (3) including in your $PATH

touch ~/.localscripts/vicontrolp
# open ~/.localscripts/vicontrolp, and pasting the commands in this file
chmod +x ~/.localscripts/vicontrolp
echo export PATH=$PATH:~/.localscripts >> ~/.bashrc
# if you use zsh, change bashrc to zshrc

:orangezellij if you want to add the same binding mentioned above.

Explaining the script

1. `fd` command

fd is an alternative to the builtin find command. It recursively lists all files within the directory.

While listing the files, it ignores the files in .gitignore.

1.1. `-E` option

Besides the files in my .gitignore there are other file formats I want to ignore. The currently excluded files are not exhaustive, you'll also want to ignore
non-text searchable files like images, videos, etc.

2. `xargs` command

This is a command that allows me to execute a new command using the inputs from stdin as argument.

2.1 What is `stdin`?

stdin is short for standard input. It is a file stream from which a program may read it's input from.
In command line scripting you'll often want to to pipe output from one command as input to another command.

2.2 What is `pipe`?

Pipe, indicated by the pipe operator: :orange[|], means to take the result of one command and pass it to the next command so it can read it as input and output a new set of data.

2.3 So how what did `fd -E ... | xargs -I{} awk ... {}` do?

The result of fd was piped to xargs. Then xargs executes awk for each files outputed by fd

2.4. `-I{}` option

This option tells xargs which character should be used to replace them with inputs from the stdin.

If for example your current directory consists the following structure:

```sh showLineNumbers=false
.
├── cskth-kt.mdx
├── images
│ └── restore_2.jpg
├── lorem-ipsum.md
└── tips-for-aspiring-professionals.mdx




doing `fd | xargs -I{} cp {} {}.bak` will be like executing the following commands (copies each file with the new file appended with `.bak`)


```sh  showLineNumbers=false frame=none
cp cskth-kt.mdx cskth-kt.mdx.bak
cp restore_2.jpg restore_2.jpg.bak
cp lorem-ipsum.md lorem-ipsum.md.bak
cp tips-for-aspiring-professionals.mdx tips-for-aspiring-professionals.mdx.bak

In our command we run the awk command for each file instead.

3. `awk` command

This program scans each line of an input file and allows you to do an action for each that matches a pattern.

3.1 `-e '...'` option

This option consists of three parts:

3.1.1. The pattern: `/^([\[\]-}{#]|[[:space:]])+$`

This regex matches when a line consists only of {,[, -, #, ], } or whitespaces

3.1.2. The action of the pattern: `{next;}`

If previous pattern matches, it tells awk to proceed to the next line and don't do any further actions.
This ultimately skips the next block which prints the important line to be piped to fzf

3.1.3. The action block: `{ print "{}:" NR ":" $0 }`

Remember that this option is still under xargs which means {} will be replaced by the filename.
Then NR in awk will print the current line of the file that's being scanned. Then $0 points to the current line.

So in our previous example, we may see the following output:

cskth-kt.mdx:1:# Heading 1
cskth-kt.mdx:2:Heading 1 content
cskth-kt.mdx:3:Heading 1 content, second line
lorem-ipsum.md:1:Lorem ipsum dolor sit amet, consectetur adipiscing elit.
lorem-ipsum.md:2:Vivamus non dapibus est, a rutrum nisi.
...

Depending of course on the contents of your files in your current directory

❗️Take note of this formatted output because we'll be explaining this later when this is piped into fzf.

3.2. `{} 2> /dev/null`

This is the parameter passed to awk command which is replaced again by xargs with the file name from fd's input.

2> /dev/null is called a stdout redirection. It simply means to redirect errors into /dev/null stream. It means to ignore any error messages
by outputting it to a blackhole or a non existent file stream: (/dev/null).

4. `fzf` command

This awesome commandline program is the heart of this script. From awk command, all contents of each file, prepended by their filename, is now piped into this program.

4.1. `--delimeter :` option

This tells fzf to use : character to separate words. This is used to separate file name, line number, and file contents again because we printed them as one line
from awk earlier.

4.2 `--preview ...` option

This tells fzf to use bat program to preview the whole file. Along with using bat, we also added parameter to highlight the current line of the current fuzzy search match.

4.3 `--preview-window ...` option

This tells fzf to scroll the preview window to include the current line that's being searched in fzf.

4.4 `--bind ...` option

Lastly, this tells fzf to do a keybinding that when enter key is pressed, open neovim and directly jump into the line number of the search match.

Conclusion

... there's really not much to conclude this with. Hopefully you'll find this script useful. ✌🏻

Tips for aspiring professionals: 4 mindset you can apply in your career and everyday life

Richard Glen Domingo — Thu, 20 Jun 2024 15:38:29 +0000

See my original post here in my blog.

4 things I realized as I reflected going through a tough day. This tips will hopefully help you prosper in your chosen field.

DEV Community: Richard Glen Domingo

How to Fuzzy Search: Finding File Names and Contents using Bash Scripting and Commandline Tools

Introduction

TLDR (just give me the bash script)

🤔 Understanding the script

Prerequisites

Explaining the script

1. `fd` command

1.1. `-E` option

2. `xargs` command

2.1 What is `stdin`?

2.2 What is `pipe`?

2.3 So how what did `fd -E ... | xargs -I{} awk ... {}` do?

2.4. `-I{}` option

3. `awk` command

3.1 `-e '...'` option

3.1.1. The pattern: `/^([\[\]-}{#]|[[:space:]])+$`

3.1.2. The action of the pattern: `{next;}`

3.1.3. The action block: `{ print "{}:" NR ":" $0 }`

3.2. `{} 2> /dev/null`

4. `fzf` command

4.1. `--delimeter :` option

4.2 `--preview ...` option

4.3 `--preview-window ...` option

4.4 `--bind ...` option

Conclusion

Tips for aspiring professionals: 4 mindset you can apply in your career and everyday life

1. As you PRACTICE, be ALWAYS on the lookout for small things that can be improved

2. Do your best to adhere to best practices

3. Keep in mind that you want to deliver value, but think in long term.

4. At the end of the day, It's all just work.

DEV Community: Richard Glen Domingo

How to Fuzzy Search: Finding File Names and Contents using Bash Scripting and Commandline Tools

Introduction

TLDR (just give me the bash script)

🤔 Understanding the script

Prerequisites

Explaining the script

1. fd command

1.1. -E option

2. xargs command

2.1 What is stdin?

2.2 What is pipe?

2.3 So how what did fd -E ... | xargs -I{} awk ... {} do?

2.4. -I{} option

3. awk command

3.1 -e '...' option

3.1.1. The pattern: /^([\[\]-}{#]|[[:space:]])+$

3.1.2. The action of the pattern: {next;}

3.1.3. The action block: { print "{}:" NR ":" $0 }

3.2. {} 2> /dev/null

4. fzf command

4.1. --delimeter : option

4.2 --preview ... option

4.3 --preview-window ... option

4.4 --bind ... option

Conclusion

Tips for aspiring professionals: 4 mindset you can apply in your career and everyday life

1. As you PRACTICE, be ALWAYS on the lookout for small things that can be improved

2. Do your best to adhere to best practices

3. Keep in mind that you want to deliver value, but think in long term.

4. At the end of the day, It's all just work.

1. `fd` command

1.1. `-E` option

2. `xargs` command

2.1 What is `stdin`?

2.2 What is `pipe`?

2.3 So how what did `fd -E ... | xargs -I{} awk ... {}` do?

2.4. `-I{}` option

3. `awk` command

3.1 `-e '...'` option

3.1.1. The pattern: `/^([\[\]-}{#]|[[:space:]])+$`

3.1.2. The action of the pattern: `{next;}`

3.1.3. The action block: `{ print "{}:" NR ":" $0 }`

3.2. `{} 2> /dev/null`

4. `fzf` command

4.1. `--delimeter :` option

4.2 `--preview ...` option

4.3 `--preview-window ...` option

4.4 `--bind ...` option