Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unix: merge many files, while deleting first line of all files

Tags:

merge

bash

I have >100 files that I need to merge, but for each file the first line has to be removed. What is the most efficient way to do this under Unix? I suspect it's probably a command using cat and sed '1d'. All files have the same extension and are in the same folder, so we probably could use *.extension to point to the files. Many thanks!

like image 969
Abdel Avatar asked Apr 11 '12 09:04

Abdel


People also ask

Which command is used to combine multiple files in Unix?

You can use w"merge.

How do I combine multiple text files into one in Linux?

To join two or more text files on the Linux command-line, you can use the cat command. The cat (short for “concatenate”) command is one of the most commonly used commands in Linux as well as other UNIX-like operating systems, used to concatenate files and print on the standard output.

Which command is used to merge lines from several files in the order they have given?

The join command allows you to merge the content of multiple files based on a common field.

Which Linux command allows you to combine two or more files together?

We know that we can use the command cat file1 file2 to concatenate multiple files.


2 Answers

Assuming your filenames are sorted in the order you want your files appended, you can use:

ls *.extension | xargs -n 1 tail -n +2 

EDIT: After Sorin and Gilles comments about the possible dangers of piping ls output, you could use:

find . -name "*.extension" | xargs -n 1 tail -n +2 
like image 142
xpapad Avatar answered Sep 25 '22 02:09

xpapad


Everyone has to be complicated. This is really easy:

tail -q -n +2 file1 file2 file3 

And so on. If you have a large number of files you can load them in to an array first:

list=(file1 file2 file3) tail -q -n +2 "${list[@]}" 

All the files with a given extension in the current directory?

list=(*.extension) tail -q -n +2 "${list[@]}" 

Or just

tail -q -n +2 *.extension 
like image 40
sorpigal Avatar answered Sep 25 '22 02:09

sorpigal