Read CSV file and save in 2 d array

Question

I am trying to read a huge CSV file in 2 D array, there must be a better way to split the line and save it in the 2 D array in one step :s Cheers

my $j = 0;
while (<IN>) 
{

    chomp ;
    my @cols=();
    @cols   = split(/,/); 
    shift(@cols) ; #to remove the first number which is a line header
    for(my $i=0; $i<11; $i++) 
    {
       $array[$i][$j]  = $cols[$i];
    }        
    $j++;    
}

Answer 1

CSV is not trivial. Don't parse it yourself. Use a module like Text::CSV, which will do it correctly and fast.

use strict;
use warnings;

use Text::CSV;

my @data;   # 2D array for CSV data
my $file = 'something.csv';

my $csv = Text::CSV->new;
open my $fh, '<', $file or die "Could not open $file: $!";

while( my $row = $csv->getline( $fh ) ) { 
    shift @$row;        # throw away first value
    push @data, $row;
}

That will get all your rows nicely in @data, without worrying about parsing CSV yourself.

Answer 2

If you ever find yourself reaching for the C-style for loop, then there's a good chance that your program design can be improved.

while (<IN>) {
    chomp;

    my @cols = split(/,/); 
    shift(@cols); #to remove the first number which is a line header

    push @array, \@cols;
}

This assumes that you have a CSV file that can be processed with a simple split (i.e. the records contain no embedded commas).