Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

exporting 20k or more records from MY SQL to CSV using php [duplicate]

I'm trying to export around 20k records from a table in mysql to csv and as expected it crashes my system, is there any alternative to doing this as to avoid crashes?

here's my current export code:

$filename2 = "csv/leads_".date("M-d-Y",time()).".csv";
            $fp2 = fopen($filename2, 'w') or die("can't open file");

            $sql2 = $sql_getcustomers;
            $res2 = mysql_query($sql2);

            // fetch a row and write the column names out to the file
            $row2 = mysql_fetch_assoc($res2);
            $line = "";
            $comma = "";
            if($row2){
                foreach($row2 as $name => $value) {
                    $line .= $comma . '"' . str_replace('"', '""', $name) . '"';
                    $comma = ",";
                }

                $line .= ",crm_group";
                $line .= "\n";
                fwrite($fp2, $line);

                // remove the result pointer back to the start
                mysql_data_seek($res2, 0);

                // and loop through the actual data
                while($row2 = mysql_fetch_assoc($res2)) {      
                    $line = "";
                    $comma = "";
                    foreach($row2 as $index => $value) {
                        $line .= $comma . '"' . str_replace('"', '""', utf8_decode($value)) . '"';
                        $comma = ",";
                    }

                    //** GET THE CRM GROUPS
                    $sql_get_group = "SELECT a.crm_group_name, b.* FROM tbl_crm_members b JOIN tbl_crm_groups a ON (a.crm_gid = b.crm_groupid) WHERE crm_uid = ".$row2["uid"];
                    $sql_get_groups = mysql_query($sql_get_group);
                    $res_get_groups = "";
                    while($sgg = mysql_fetch_object($sql_get_groups)) $res_get_groups .= $sgg->crm_group_name.";";
                    $line .= ",".trim($res_get_groups, ";");
                    $line .= "\n";
                    fwrite($fp2, $line);    

                }
                fclose($fp2);
like image 536
magicianiam Avatar asked Oct 04 '13 17:10

magicianiam


People also ask

How to export data from MySQL to CSV in PHP?

Create a CSV file in PHP and save data in it. Export MySQL data and download it in a CSV file using PHP. To store the data, a table needs to be created in the database. The following SQL creates a members table with some basic fields in the MySQL database.

How do I export my data to a CSV file?

Use the following command to export to a CSV file, and add a timestamp for the time the file was created: It is often convenient to add column headers to the output file to better identify and analyze the data. To do this, you must use the UNION statement. (SELECT 'columnHeading', ...) UNION (SELECT column, ...

How to export table data to CSV file in WordPress?

The Export link will be provided at the top of the table that navigates to the exportData.php file to export table data to CSV file. In the exportData.php file, the following works are processed. Retrieve data from the MySQL database. Create a file pointer using fopen () function. Specify the header columns and put into the CSV file.

How do I export data from the database?

Initially, all the member’s data is fetched from the database and listed in a tabular format. An EXPORT button is placed at the top of the data list. By clicking the Export button, the data is exported from the database and allow to download on a local drive as a CSV file.


1 Answers

Why not let mysql do it?

SELECT id, name, email INTO OUTFILE '/tmp/result.csv'
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '\\'
LINES TERMINATED BY '\n'
FROM users WHERE 1

http://ariejan.net/2008/11/27/export-csv-directly-from-mysql/

like image 80
Anton Bessonov Avatar answered Sep 22 '22 16:09

Anton Bessonov