Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Combining 2 CSV files

Tags:

php

I'm trying to combine two CSV files in PHP. I'm looking for perfect method. Here's my code so far:

$one = fopen('data5.csv', 'r');
$two = fopen('userdata.csv', 'r');

$final = fopen('final_data.csv', 'a');

$temp1 = fread($one, filesize("data5.csv"));
$temp2 = fread($two, filesize("userdata.csv"));

fwrite($final, $temp1); 
fwrite($final, $temp2);
like image 257
no_freedom Avatar asked Mar 24 '11 10:03

no_freedom


People also ask

Can you combine two CSV files?

Option 1: Command Prompt If you are a Windows user, you can use the built-in Command Prompt to combine CSV files. Command Prompt is a text interface for your computer. You can type simple commands to merge files. First, put all of your CSV files in a folder and copy the full path of your folder.


2 Answers

I will give you a solution to use if you have big CVSs and you don't want to use much of your machine's RAM (imagine each CSV is 1GB, for example).

<?php
function joinFiles(array $files, $result) {
    if(!is_array($files)) {
        throw new Exception('`$files` must be an array');
    }

    $wH = fopen($result, "w+");

    foreach($files as $file) {
        $fh = fopen($file, "r");
        while(!feof($fh)) {
            fwrite($wH, fgets($fh));
        }
        fclose($fh);
        unset($fh);
        fwrite($wH, "\n"); //usually last line doesn't have a newline
    }
    fclose($wH);
    unset($wH);
}

Usage:

<?php
joinFiles(array('join1.csv', 'join2.csv'), 'join3.csv');

Fun fact:

I just used this to concat 2 CSV files of ~500,000 lines each. It took around 5seconds and used 512kb of memory.

Logic:

Open each file, read one line and then write it to the output file. Yes, it may be slower writing each line rather than writing a whole buffer, but this allows the usage of heavy files while being gentle on the memory of the machine. At any point, you are safe because the script only reads on line at a time and then writes it.

Enjoy!

like image 71
Bogdan Constantinescu Avatar answered Sep 17 '22 13:09

Bogdan Constantinescu


How about...

file_put_contents('final_data.csv',
    file_get_contents('data5.csv') .
    file_get_contents('userdata.csv')
);

Note that this loads the entire files into PHP memory though. So, if they are big, you may get memory_limit issues.

like image 33
Sander Marechal Avatar answered Sep 18 '22 13:09

Sander Marechal