DEV Community

Cover image for Append text in each line in a large CSV (unix style)
Leylow Lujuo
Leylow Lujuo

Posted on

1 3

Append text in each line in a large CSV (unix style)

Say you have a 10GB csv file and you want to add a text in the end of each line.

Obviously, opening it in vim work cut it

oh will it? ... let me confirm ... I tried and vim crushed :( ...

"So, what do we do?" I asked my conscious self while googling and stack-overflow-ing with 10+ tabs opened.

Viola! one of my stack-overflow tab came through and the answer was sed - a stream editor.

sed -r 's/text_to_find/text_to_replace/' input_file > output_file
# example
sed -r 's/\r/|the_text\r/' big_file.csv > output.csv

Basically, what it does here is;

-r this option represent extended regular expression

's/**\r**/|**the_text\r**/' the \r represent EOL, so replace EOL with the_text plus EOL(so that the output won't be a single line file)

The rest i.e... input and output file is self explanatory.

Writing is exhausting, I'm done. I have being told to be positive, so i'm scratching this and the line below.

Now i have to search for a cover image, smh.

Thanks for your one minute, hopefully I will write again.

Image of Quadratic

Free AI chart generator

Upload data, describe your vision, and get Python-powered, AI-generated charts instantly.

Try Quadratic free

Top comments (0)

Sentry image

See why 4M developers consider Sentry, “not bad.”

Fixing code doesn’t have to be the worst part of your day. Learn how Sentry can help.

Learn more

👋 Kindness is contagious

Dive into this thoughtful article, cherished within the supportive DEV Community. Coders of every background are encouraged to share and grow our collective expertise.

A genuine "thank you" can brighten someone’s day—drop your appreciation in the comments below!

On DEV, sharing knowledge smooths our journey and strengthens our community bonds. Found value here? A quick thank you to the author makes a big difference.

Okay