Delete empty lines in csv file

Question

I have a file with 4 million of lines, every line ends with the char $, but I mistakenly add a new line after the the line delimiter while scraping a website, so right now it is looking like this:

fist name, last name, phone, address, postal code, city, region,$

$

fist name, last name, phone, address, postal code, city, region,$

$

the new line '$' only shows up of course if I use :set list, but I'm trying to use this file for a bulk insert in mysql and I'm having problems with it now.

I would like to change the file to:

fist name, last name, phone, address, postal code, city, region,$

fist name, last name, phone, address, postal code, city, region,$

How can I do this? with sed or awk or even vi ? looked up around and what I found is not really applying to this case.

please don't take in consideration the extra empty line shown above.

Thanks in advance

Answer 1

To remove blank lines with sed:

sed -i '/^$/d' yourfile.csv

To remove lines consisting of a single $:

sed -i '/^$$/d' yourfile.csv

Most versions of sed support the -i switch; if yours does not you will need e.g. sed '/^$$/d' yourfile.csv > newfile.csv.

Removing blank lines with white space is more complicated. This usually works:

sed '/^ *$/d' yourfile.csv

If this is not sufficient, try checking also for tabs. For older sed's, this will work:

sed '/^[ X]*$/d' yourfile.csv

where X here a tab, entered via Control-V Tab.

Newer sed's will take a [ \t\r]* or \s* or [[:space:]]*, sometimes requiring a -E switch.