How to remove header and footer records of a flat file using UNIX shell script?

Question

I am having a flat file as give below. How do I delete the header and footer from the file using UNIX shell script and rewrite the same file.

9 20050427 HEADER RECORD
0000000 00000 000000000 123456 00 654321 DATARECORD
0000000 00000 000000000 123456 00 654321 DATARECORD
0000000 00000 000000000 123456 00 654321 DATARECORD
0000000 00000 000000000 123456 00 654321 DATARECORD
6 20050427 TRAILER RECORD

Thanks, Arun

Answer 1

... and with sed:

As @Baramin noted: the least amount to type is sed '1d;$d', here is how it works:

By line number

sed -i'' -e '1d' -e '$d' yourfile

1d deletes first line $d deletes last line.

Or by pattern

sed -r -i -e '/^[0-9] [0-9]{8} HEADER RECORD$/d' \
          -e '/^[0-9] [0-9]{8} TRAILER RECORD$/d' yourfile

-r is needed for the {8} extended regular-expression.

Or both

If you are super pedantic, and want to cover your ass in the most thorow way:

sed -r -i.bak -e '1{/^[0-9] [0-9]{8} HEADER RECORD$/d}' \
              -e '${/^[0-9] [0-9]{8} TRAILER RECORD$/d}' yourfile

---

The -i'' will change yourfile in-place. Drop it if you want to store the output in another file (e.g. with > outputfile at the end of your line).

Answer 2

tail -n +2 filename | head -n -1

don't have my unix box to test on so those numbers might be 1 or 2, i can't remember if they are bot 1, both 2 or what but i ise this all the time (i just experement right before i run the cmd to see if its 1 or two... the tail ... should remove the first line and the head ... should remove the last

Answer 3

This is ugly, but it seems to work, at least on your input:

f='test.txt'; g=`wc -l $f`; h=`echo $g | cut -d ' ' -f1`; head -n $((h-1)) $f | tail -n $((h-2))

f is the name of your file. I couldn't work out a quicker way to get rid of the filename from the output of wc. Someone should be able to beat this.

If you want to rewrite the same file, just redirect the output of the command:

f='test.txt'; g=`wc -l $f`; h=`echo $g | cut -d ' ' -f1`; head -n $((h-1)) $f | tail -n $((h-2)) > $f