Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove lines which are between given patterns from a file (using Unix tools)

Tags:

unix

lines

I have a text file (more correctly, a “German style“ CSV file, i.e. semicolon-separated, decimal comma) which has a date and the value of a measurement on each line.
There are stretches of faulty values which I want to remove before further work. I'd like to store these cuts in some script so that my corrections are documented and I can replay those corrections if necessary.

The lines look like this:

28.01.2005 14:48:38;5,166
28.01.2005 14:50:38;2,916
28.01.2005 14:52:38;0,000
28.01.2005 14:54:38;0,000
(long stretch of values that should be removed; could also be something else beside 0)
01.02.2005 00:11:43;0,000
01.02.2005 00:13:43;1,333
01.02.2005 00:15:43;3,250

Now I'd like to store a list of begin and end patterns like 28.01.2005 14:52:38 + 01.02.2005 00:11:43, and the script would cut the lines matching these begin/end pairs and everything that's between them.

I'm thinking about hacking an awk script, but perhaps I'm missing an already existing tool.

like image 356
Florian Jenn Avatar asked Jan 03 '10 22:01

Florian Jenn


People also ask

How do you remove lines from a file in Unix?

To delete a line, we'll use the sed “d” command. Note that you have to declare which line to delete. Otherwise, sed will delete all the lines.

How do you delete a line containing a pattern in Unix?

To begin with, if you want to delete a line containing the keyword, you would run sed as shown below. Similarly, you could run the sed command with option -n and negated p , (! p) command. To delete lines containing multiple keywords, for example to delete lines with the keyword green or lines with keyword violet.

How do I delete a range of lines in Linux?

The sed command can remove the lines of any range. For this, we just have to enter 'minimum' and 'maximum' line numbers. In this example, we will remove the lines ranging from 4 to 7 numbers. After removing these ranges of lines, our file will look like this.

How do you remove multiple lines in Unix?

Deleting Multiple LinesPress the Esc key to go to normal mode. Place the cursor on the first line you want to delete. Type 5dd and hit Enter to delete the next five lines.


1 Answers

Have a look at sed:

sed '/start_pat/,/end_pat/d'

will delete lines between start_pat and end_pat (inclusive).

To delete multiple such pairs, you can combine them with multiple -e options:

sed -e '/s1/,/e1/d' -e '/s2/,/e2/d' -e '/s3/,/e3/d' ...
like image 103
Alok Singhal Avatar answered Oct 17 '22 01:10

Alok Singhal