I have two set of folders that have files with the same filenames and structure. The folder structure is something like this:
\outputfolder\
|---\folder1\
| |---file1.txt
| |---file2.txt
|
|---\folder2\
|---file1.txt
|---file2.txt
So what I need to do is to combine (append) all the files with the same name in these folders (file1.txt with file1.txt etc.) into another file inside the outputfolder. After getting these combined files I also need to create a tar.gz file from all of these combined files.
How can I accomplish this in a Linux based command line environment? The folder name (folder1 and folder2 etc) is variable so this needs to be given but the files need not and it should automatically combine all the files with the same name.
Also, these files have headers for column names, so I would need to remove that as well while appending.
Here's some code to get you started
topdir=outputfolder
dir1=folder1
dir2=folder2
for f in $topdir/$dir1/*.txt
do
outf=$topdir/`basename $f .txt`-concat.txt
cp $f $outf
sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf
done
tar czf foo.tar.gz $topdir/*-concat.txt
Edit: added the part removing the header of the 2nd file.
find . -name 'file1.txt' | xargs cat >file1_concat.txt
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With