Combining two files in different folders in Linux

Question

I have two set of folders that have files with the same filenames and structure. The folder structure is something like this:

\outputfolder\
 |---\folder1\
 |      |---file1.txt
 |      |---file2.txt
 |
 |---\folder2\
        |---file1.txt
        |---file2.txt

So what I need to do is to combine (append) all the files with the same name in these folders (file1.txt with file1.txt etc.) into another file inside the outputfolder. After getting these combined files I also need to create a tar.gz file from all of these combined files.

How can I accomplish this in a Linux based command line environment? The folder name (folder1 and folder2 etc) is variable so this needs to be given but the files need not and it should automatically combine all the files with the same name.

Also, these files have headers for column names, so I would need to remove that as well while appending.

Answer 1

Here's some code to get you started

topdir=outputfolder
dir1=folder1
dir2=folder2

for f in $topdir/$dir1/*.txt
do
    outf=$topdir/`basename $f .txt`-concat.txt
    cp $f $outf
    sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf
done

tar czf foo.tar.gz $topdir/*-concat.txt

Edit: added the part removing the header of the 2nd file.

Answer 2

find . -name 'file1.txt' | xargs cat >file1_concat.txt