Combining awk and vim to get real power

Foreword

After the initial post of this little, "how to", @Sundeep commented:

why not use awk itself to insert the snippet? since you are using gawk, you can also use -i inplace if you want inplace editing

I don't remember exactly, but I think you can use silent to avoid having to use Enter manually, check these: ...

I will explain the scenario, my old solution and then the gawk solution

The Scenario

I have about 500 markdown files on my personal vimwiki, for years I have been making notes to help myself and others through Linux/vim commands. Recently I started using pandoc to convert my markdown files to html and LaTeX.

using vim + gawk

I have two mappings on my ~/.config/init.vim that makes it easy to convert my current documment to html and pdf. During the process of convertion pandoc warns about the lack of the title, and this brought me here.

if !exists('*PandocMappings')
    function! PandocMappings()
        " NOTE: <F16> equals to Shift-F4
        noremap <F4> :! pandoc '%:p' -o /tmp/'%:p:t'.html --template=easy_template.html --toc && firefox /tmp/'%:p:t'.html &<CR><CR>
        noremap <F16> :! pandoc '%:p' -o /tmp/'%:p:t'.pdf --pdf-engine=pdflatex --toc && evince /tmp/'%:p:t'.pdf &<CR><CR>
    endfun
endif

augroup my_markdown
    autocmd!
    autocmd FileType markdown call PandocMappings()
augroup END

My markdown files lack metadata, something like this:

--------
file: the file name
lang: pt-BR
author: Sergio Araujo
date: abr 03, 2019
created: abr 03, 2019
abstrac: This aims...
--------

After adding manually the metadata in about eight of them I realized that I had better tools and the knowledge (including web search) to solve my problem. The metadata should start at first line.

Finding out the files with no metadata

Using awk, more specificly gnu awk "gwak" printing the list of files who have the metadata is simple

gawk '(FNR==1 && $1 ~ /^---/) {print FILENAME}' *.md

FNR==1  ...................... the first line equals to the first register
&& ........................... it allows us to test a second condition
~  ........................... match regular expression
/^---/ ....................... the regular expression itself

The command above showed me the files I already had inserted the metadata, inverting the regex is very simple, just use exclamation. So the list of files that did not had the metatada can be printed with:

gawk '(FNR==1 && $1 !~ /^---/) {print FILENAME}' *.md

Combining the awk result with vim

Using the shell Command Substitution we can deliver to vim the list we wanted.

vim $(gawk '(FNR==1 && $1 !~ /^---/) {print FILENAME}' *.md)

The vim/neovim argdo command

When you open vim with something like:

vim *.md

All the opened files are in the arglist, so we can use something like:

:argdo command

A UltiSnippet snippet to insert the header

snippet headermd "markdown header" w
--------
file: `!p snip.rv = snip.fn`
lang: pt-BR
author: `!v g:snips_author`
date: `!v strftime("%b %d, %Y")`
created: `!v strftime("%b %d, %Y")`
--------
endsnippet

the vim function that can be called by argdo to inserts the snippet

fun! InsertHeaderMd()
    0pu_
    exe "normal! iheadermd\<C-r>=UltiSnips#ExpandSnippet()\<CR>"
endfun

0pu_  ............................ On the line zero insert black hole register (blank line)
exe normal! ...................... normal command
iheadermd ........................ enters insert mode and types headermd
\<C-r>= .......................... expression register
UltiSnips#ExpandSnippet()\<CR> ... the UltiSnippet trigger to call the snippet

Finishing up everything

We do not even have put the function (just the snippet) on the configuration files, you can copy the function to the clipboard and type this on vim:

:@+

Now with all files opened you can do:

:silent argdo call InsertHeaderMd() 
:silent argdo update

Thanks to @Sundeep I added the silent option which allowed me run the command without hiting Enter a bunch of times. And he also suggested awk as a solution

Using a vim macro with argdo

Opening the files without metadata

vim $(gawk '(FNR==1 && $1 !~ /^---/) {print FILENAME}' *.md)

Creating a vim macro

:let @a=''
:let @a="ggO---\<Return>title: \<C-r>=expand('%:t')\<CR>\<Return>author: Sergio Araújo\<Return>abstract:\<Return>---\<Esc>"

Explanation

let @a='' ............................ delete any marcro 'a'
ggO .................................. got to the first line and open a line above
---\<Return> ......................... insert three dashes and a new line
\<C-r>=  ............................. starts a expression register
expand('%:t') ........................ expands the filename "t = tail" 
\<Esc> ............................... goes back to the normal mode

Running the argdo command plus a macro

:set hidden ............................ allows to go to the next file without saving

:silent argdo execute "normal! @a" | update 

:silent ................................. does not show any messages

The gawk solution

This solution came from a @Sundeep comment

  var=$(date "+%b %m, %Y")
  awk -i inplace -v date="$var" 'BEGINFILE {print "--\nfile: "FILENAME"\ndate: "date"\nabstract: \n---"}; (FNR==1 && $1 !~ /^---/) {print}' *.md 

Job done!

Explaining the awk long command...

-i inplace ..................... makes possible edit files directly in awk
-v date="$var" ................. captures the content of var=$(date "+%b %m, %Y")
BEGINFILE ...................... insets some strings "between quotes"
FILENAME ....................... gawk variable that returns the file name
FNR==1  ........................ the first line equals to the first register
&& ............................. it allows us to test a second condition
~  ............................. match regular expression
/^---/ ......................... the regular expression itself

The next step will be using a variable using shell Here Documents. If you guys can help me with this I would thank you a lot!

Could it be done with sed?

I know that sed can also insert a header in a bunch of files, but I should have write a shell script to get each file name or use -V sed option. The solution I figured out was good for my problem, and it also inspired me enough to share this ideas with you. Feel free to comment and suggest alternatives or improvements. (Please fix my typing, I am not a native English speaker).

References:

• https://stackoverflow.com/questions/32481877/what-is-nr-fnr-in-awk
• https://unix.stackexchange.com/questions/440088/what-is-command-substitution-in-a-shell
• https://www.cyberciti.biz/faq/how-to-print-filename-with-awk-on-linux-unix/

Comments

[Sundeep]

why not use awk itself to insert the snippet? since you are using gawk, you can also use -i inplace if you want inplace editing

I don't remember exactly, but I think you can use silent to avoid having to use Enter manually, check these:

• vimcasts.org/episodes/using-argdo-...
• stackoverflow.com/questions/890802...
• vi.stackexchange.com/questions/788...

[Sérgio Araújo]

Thanks @Sundeep for your tips! I will update my post. But actually the real purpuse of it was reached, since many people will learn more about awk, vim, and of course, I will learn more with your great suggestions.

[Sérgio Araújo]

Hi @Sundeep, I have made some changes on my article, now it shows a pure awk solution, a vim macro solution + awk. Both brings us more ideas for the future needs.