Remove first line in text file without allocating memory for entire text file

Question

I have a very large text file and all I need to do is remove one single line from the top of the file. Ideally, it would be done in PHP, but any unix command would work fine. I'm thinking I can just stream through the beginning of the file till I reach \n, but I'm not sure how I do that.

Thanks, Matt Mueller

Answer 1

you can use a variety of tools in *nix. A comparison of some of the different methods on a file with more than 1.5 million lines.

$ wc -l < file4
1700589

$ time sed -n '2,$p' file4 > /dev/null

real    0m2.538s
user    0m1.787s
sys     0m0.282s

$ time awk 'NR>1' file4 > /dev/null

real    0m2.174s
user    0m1.706s
sys     0m0.293s

$ time tail -n +2 file4 >/dev/null

real    0m0.264s
user    0m0.067s
sys     0m0.194s

$time  more +2 file4 > /dev/null

real    0m11.771s
user    0m11.131s
sys     0m0.225s

$ time perl -ne 'print if $. > 1' file4 >/dev/null

real    0m3.592s
user    0m3.259s
sys     0m0.321s

Answer 2

sed -i -e '1d' file will do what you want.

• -i indicates "in-place"
• -e means "evaluate this expression"
• '1d' means, delete the first line