Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Powershell / Perl : Merging multiple CSV files into one?

I have the following CSV files, I want to merge these into a single CSV

01.csv

apples,48,12,7
pear,17,16,2
orange,22,6,1

02.csv

apples,51,8,6
grape,87,42,12
pear,22,3,7

03.csv

apples,11,12,13
grape,81,5,8
pear,11,5,6

04.csv

apples,14,12,8
orange,5,7,9

Desired output:

apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,87,42,12,81,5,8,,,
pear,17,16,2,22,3,7,11,5,6,,,
orange,22,6,1,,,,,,5,7,9

Can anyone provide guidance on how to achieve this? Preferably using Powershell but open to alternatives like Perl if that's easier.

Thanks Pantik, your code's output is close to what I want:

apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,87,42,12,81,5,8
orange,22,6,1,5,7,9
pear,17,16,2,22,3,7,11,5,6

Unfortunately I need "placeholder" commas in place for when the entry is NOT present in a CSV file, e.g. orange,22,6,1,,,,,,5,7,9 rather than orange,22,6,1,5,7,9

UPDATE: I would like these parsed in order of the filenames, e.g.:

$myFiles = @(gci *.csv) | sort Name
foreach ($file in $myFiles){

regards ted

like image 793
ted Avatar asked May 13 '11 07:05

ted


People also ask

How do I combine files in PowerShell?

Use Out-File to Concatenate Files Using PowerShell The Out-File cmdlet sends the output to a file. If the file does not exist, it creates a new file in the specified path. To concatenate files with Out-File , you will need to use the Get-Content cmdlet to get the content of a file.

How do I merge multiple CSV files in Windows 10?

To merge multiple CSV files into one, you can use the Command Prompt on Windows 11/10 computer. To do so, you need to paste all the CSV files to one folder, open Command Prompt in that folder, and enter this command: copy *. csv newfile. csv.


1 Answers

Here is my Perl version:

use strict;
use warnings;

my $filenum = 0;

my ( %fruits, %data );
foreach my $file ( sort glob("*.csv") ) {

    $filenum++;
    open my $fh, "<", $file or die $!;

    while ( my $line = <$fh> ) {

        chomp $line;

        my ( $fruit, @values ) = split /,/, $line;

        $fruits{$fruit} = 1;

        $data{$filenum}{$fruit} = \@values;
    }

    close $fh;
}
foreach my $fruit ( sort keys %fruits ) {

    print $fruit, ",", join( ",", map { $data{$_}{$fruit} ? @{ $data{$_}{$fruit} } : ",," } 1 .. $filenum ), "\n";
}

Which gives me:

apples,48,12,7,51,8,6,11,12,13,14,12,8
grape,,,,87,42,12,81,5,8,,,
orange,22,6,1,,,,,,,5,7,9
pear,17,16,2,22,3,7,11,5,6,,,

So do you have a typo for grape or i have misunderstood something?

like image 163
gangabass Avatar answered Nov 14 '22 23:11

gangabass