I like all the Vim tutorials on Dev.to. They're great! They're good starting points. Please go and read them all!
But...
I'd like to tell you all the secret teaching of Vim. Bit of a sidetrack. You know, the stuff you get in the 36th chamber of Shaolin or whatever, because for some reason we love mixing up all our software development metaphors with martial arts and/or Zen Buddhism.1
Let me tell you about the Unix way...
Unix Philosophy
No, I'm not going to explain what Unix is. You have Google for that. Here's a link to the Wikipedia page on 'Unix philosophy', which is pretty long so let's get to the point, eh?
In the words of the Ancient Grand Master Douglas McIlroy:
Write programs that do one thing and do it well.
Which is easy to say but not easy to do. What if you want to do more than one thing?
Pipes
Pipes are the jewel of a Unix system. To me, its crowning glory. You send the output of one program as the input to another program - just like putting bits of hose pipe together and passing a stream of water through.
Say I wanted a random word with 'bob' in it somewhere. On a Unix like machine you will find a file with a lot of words in it - a dictionary. Mac and BSD systems have it at /usr/share/dict/words.
We can use the program cat to show it in a terminal:2
cat /usr/share/dict/words
But now we need to find all the words with 'bob' in ... well, we could open the file with grep
grep bob /usr/share/dict/words
but I find it more intuitive to pass the output of cat into grep using a pipe, which we write with the pipe symbol: |3
cat /usr/share/dict/words | grep bob
Well that gives me all the words with 'bob' in - how will I get a random one? There are a few ways of doing this, but here's one: I have the program shuf on my computer, which shuffles the lines you send in:
cat /usr/share/dict/words | grep bob | shuf
and now I just want one of the lines - it doesn't matter which one. Why not just the first one then? head will do - it can give us just the first line from the input:
cat /usr/share/dict/words | grep bob | shuf | head -1
and behold
> cat /usr/share/dict/words | grep bob | shuf | head -1
bobolink
> cat /usr/share/dict/words | grep bob | shuf | head -1
nabobish
What does that mean? I can use some other programs to help - I could open (at least on OSX) a URL with that word in it, using a handy program called parallel:
cat /usr/share/dict/words | grep bob | shuf | head -1 | parallel open https://en.wiktionary.org/w/index.php?title={}
and then... and then... and then...
The important lesson is this: we didn't write a new program. We didn't download a new random bob word generator. What we did was use a series of programs that do one thing (and one thing well) and glued them all together to get what we wanted.
But you promised us Vim!
Yes, yes. Vim. I've used Vim since I started developing - I used to write marketing emails using it to make writing marketing emails just a little bit more interesting (and therefore bearable). We always bang on about the keybindings in Vim - the magic of hjkl - but you can add those to almost any text editor these days.
So what do I think is great about Vim?
This:
:%!
For the Vim novitiates among us: : indicates we're entering a command. % is a reference to all the lines in the current buffer. And ! indicates we're going to run a command in the shell.
The above will pipe your current buffer through a shell command and replace the contents. Try it!
:%! shuf
Well that was fun, if highly destructive. But it can be a lot more useful. Want to reformat all the JSON in your buffer? jq is your friend:
:%! jq '.'
or maybe you'd like Python to do the same thing:
:%! python -m json.tool
view some Markdown as HTML? Try pandoc:
:%! pandoc
You can do it with a range of lines, or a visual selection too - as always with Vim, help is available:
:h !
Now you can leverage other people's programs to extend your editor without installing extra modifications, or plugins, or using a new editor as you would with things like VSCode, Atom or Sublime. And if you find something useful, and use it over and over again, you can write a bit of VimL and to make the command easier to run.
Another advantage - by learning some of the simple, do-one-thing oriented programs that are available in a Unix environment, you'll be able to use them over and over again in more and more programs that you write.
Over to you
This becomes very real when you hit a problem that you've not got a tool for. For me, it was when I needed to translate some large JavaScript objects into JSON. A problem I couldn't see an easy way to complete in the editor4.
So I wrote a short(ish) NodeJS script that takes a stream of JavaScript and outputs a stream of JSON. Not beautiful, but effective.
Having written this once, with a pipe-friendly interface, I could now do this to a JavaScript object in a Vim buffer:
:%! js2json
or in the shell:
cat some.js | js2json | curl -X POST -d @- http://gimme-json.com
or maybe I'll never use it again.
But it's there. I've extended my editor by leveraging its ability to integrate with the programs I write. Thanks Vim! Thanks Unix!
Vi-Nature is Unix-Nature
Vim is not great because it impresses your friends with your knowledge of esoteric keystrokes,5 or because your elite touch typing skills allow you to churn out x-ity-x-billion characters per second by never taking your hands off the keyboard. These are fun things but they're shallow things. We should not obsess over them, and we should not be smug about knowing them.
Vim is great because it's everywhere - if you know Vim/vi then you've got an editor on most Unix systems you can use. It's great because you can integrate it into Unix tooling - and that's great because you can write those tools yourself because they're not that complicated and you're a programmer!
You'll see the same idea in other editors too. In the words of Master Wq:
Vim is not permanent. nvi is not permanent. vi itself is not permanent, only vi-nature. Emacs has vi-nature, nano has vi-nature, even Notepad has vi-nature. You narrow your sights, you grow attached [...]. You must leave. Come back when you have mastered Emacs.6
Where you can do the same with the simple Emacs command:
C-x h C-u M-| js2json
Happy hacking!
-
It's all ninjas and koans, or its woodworking and 'craftmanship'. ↩
-
Strictly speaking this is an abuse of
cat. ↩ -
The idea was McIlroy, but the
|symbol was all Ken Thompson. Imagine that first day of using it - pipemania! ↩ -
A hard way to solve it would have been to write a macro. That would have been throwaway and probably as time consuming. ↩
-
You must all try
ggg?Gright now. ↩ -
Only two editors missing, and you should try them both:
edandacme. ↩
Top comments (35)
The (notionally) more portable — and less abusive of felines — method would be something like:
Thankfully,
shuflets you limit the amount of lines output ...meaning you can save yourself the scads of resources of callinghead! =)Or, if you want to avoid helpers like
xargsorparallelaltogether:If you're going to abuse a
cat, do so by trying out multiple skinning-methods. =)All very true, and very good!
🤣
What I tend to use more often than
:%!is to actually highlight a number of lines usingV(visual line) and then apply an operation by simply typing:and then!(vim automatically inserts the range specifier'<,'>for you), followed by your command. I typically do this withsortto arrange import sections in particular.Yup - me too. And a quick look at
:h !will show you how to pass through motions and line ranges to an external program.The possibilities are ... well, not endless. But there are a lot of them.
If you enjoy doing this sort of thing, you should really take Acme out for a spin.
Saying "the possibilities are finite!" is more accurate but less fun.
Another thought: although
findimplementations can be bloated with extra functionality, they always play nicely with pipes. That's the one rule that nobody wants to break, because it's so damn useful!I think you might be looking for the second part of the Unix philosophy:
Composition is important... but reading
I always google before using
find. But it is a powerful tool. For simpler tasks, I just usels -1 GLOBorls -1 | grep PAT.One task where
findreally helped was finding and removing broken symlinks in a directory with over 10 million files.I start to wonder whether the whole 'do one thing' idea is good because it's easier to remember the 'one thing' program's simple and focused interface, and so compose it with other 'one thing' programs, than it is to remember the large and complicated interface of things like
find.Because I also have to look up how to use
findevery time I have to use it - but the piping oflstogrepis intuitive.In other words, in order to get all the benefits of composition, don't you need the simplicity of 'do one thing'?
I always google before using
find. But it is a powerful tool. For simpler tasks, I just usels -1 GLOBorls -1 | grep PAT.One task where
findreally helped was finding and removing broken symlinks in a directory with over 10 million files.Great article, showing an entire extra dimension for capabilities of a good editor.
RE the Emacs example, let me explain for posterity. This may look scary:
But it's actually built out of commands Emacs regulars would know well.
C-x hmeans "select all" (more accurately, themark-whole-buffercommand).C-uis a "prefix argument", applying to the next command you issue; it's something used frequently.M-|isshell-command-on-region; it'll take what you selected and pipe it into a shell command. Without a prefix argument, the output would end up in another Emacs buffer; with a prefix argument, it replaces the selection.(And, of course,
C-xis "control + x",M-|is "alt + |" in Emacs-speak.)Cheers!
Many Unix programs have deviated from the 'Unix philosophy', the best example being
catwhich is rarely if ever used to concatenate files any more.Take a look at cat -v considered harmful where Rob Pike writes very well about this.
Finally someone
What do you mean by this?
Unix Philosophy is very similar to Functional Programming philosophy. They overlap in many ways e.g. small programs vs small functions, do one thing vs pure (no side-effect) functions etc. However sometimes overall productivity both short and long term as well as efficiency is better if we not primitively and blindingly follow the utopic idea (Unix-way or FP). Example: vim follows Unix-philosophy and VSCode (or WebStorm etc.) breaks it. The former while it is a cool concept, generally is inefficient because it has a very, very steep learning curve. Regarding FP: Haskell and friends are generally less efficient than JS because you have to become a f***ing (sorry, I lack vocabulary to express myself) mathematician to start gain something from it.
While I generally love the Unix (and FP) Philosophy, it should be taken with a grain of salt (i.e. applied to appropriate problems e.g. low-level programs), just like any idea.
You are talking about a purely functional language. The paradigm itself is way easier to get into, is just a matter of wanting to learn something. Simple as that. Functional programming doesn't need to be understood just by mathematical references.
And then you go and say "I love" ? Really?
Thanks for the tutorial! Made me seek whether there's an extension (atom.io/packages/pipe) for Atom which could enable the same way of thinking, and I think I found one which I plan to try soon.
Yes, this is like saying that Javascript is the best programming language that runs in the browser. While it's true, it tells us nothing about the quality of Javascript's design (or lack thereof in many aspects).
I would argue against using Vim to preach the Unix philosophy for non-believers. It could backfire, and rightly so.
Vim gets a special pass because it's a fascinating part of computer history. But if it was invented today, nobody would be insane enough to use it.
Really, users should not have to try hard to understand a piece of software. We understand today that it's the software that has to try hard to understand its users.
The Unix philosophy is great though! But it is about having simple core concepts that you can combine ad nauseam. Your paragraphs on pipes is a much better illustration of it!
Not a bad point, but I think you're a little mean to call the most popular editor for sysadmins/devops a 'part of computer history'. It's still pretty popular for web developers.
So... I lay down a challenge to you: can you name another editor which has a similar command to call out to the shell and replace the current buffer/file with the contents? i.e. I can extend the editor using external programs.
I have various ways to do that in Jetbrains IntelliJ, like this one
jetbrains.com/help/idea/settings-t...
Reading again my comment, it's true that my tone was a bit mean. I used vim a lot and found it fascinating. I think we have better tools now, but that's not vim's fault, that's progress.
If you are interested about Vim's history, this link is your friend.
Good writing BTW
That's a nice piece - thanks. Just read his piece on Lisp which I thought was very well put together.
Why ?
Since you ask...
I see, I thought you refer to put together with the Vim's link, my bad english background ! That was I asked you, but thanks for your response, I'm going to read the Lisp's link... and of course, I'm agree with "well referenced and researched"
It sounds like what you really want is acme, not vi.
Yup, acme is the ... well, it's the acme of this idea. My only personal issue with it is the mouse, but that's probably practice.
On the subject of acme, did you see this:
acme/acme2k: A lesson in sane IDE defaults.
tux0r
"Stop Piping Cats" ibm.com/developerworks/aix/library...
Hmmm... never realized that
cattakes more time!That said, where he goes:
I prefer:
But I'm not sure why. It's either because I don't have to remember where to put the file argument to
grep. Or because it's my go-to 'save acat' today technique. Or because I'm pretentious.</usr/share/dict/words>is the most frightening closing tag I think I've ever seen.I think the most important part of living up to the Unix philosophy is abiding by the core tenet: text is the universal interface.
Looks nice in my blogsite. Or in short, simply show off.