Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Merge two large CSV files with PHP

Tags:

php

csv

I want to merge two large CSV files with PHP. This files are too big to even put into memory all at once. In pseudocode, I can think of something like this:

for i in file1
  file3.write(file1.line(i) + ',' + file2.line(i))
end

But when I'm looping through a file using fgetcsv, it's not really clear how I would grab line n from a certain file without loading the whole thing into memory first.

Any ideas?

Edit: I forgot to mention that each of the two files has the same number of lines and they have a one-to-one relationship. That is, line 62,324 in file1 goes with line 62,324 in file2.

like image 537
Jason Swett Avatar asked Nov 28 '25 20:11

Jason Swett


1 Answers

Not sure what operating system you're on, but if you're using Linux, using the paste command is probably a lot easier than trying to do this in PHP.

If this is a viable solution and you don't absolutely need to do it in PHP, you could try the following:

paste -d ',' file1 file2 > combined_file
like image 75
mfonda Avatar answered Nov 30 '25 09:11

mfonda



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!