import csv file from external FTP Site

Question

I have the below script to import data from a csv file on my server, the script works correctly.

I need to change the location of the file from my server to an FTP server which requires authentication. The csv file name on the FTP server will cahnge according to the time stamp it was generated. After importing into the MySQL database, the file on the FTP server needs to be deleted.

I then need to schedule this job with cron jobs to run every 5 minutes.

Any assistance will be appreciated.

Thanks.

---

<?php

/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$databasehost = "localhost";
$databasename = "dbname";
$databasetable = "dbtable";
$databaseusername ="username";
$databasepassword = "password";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filenamewithtimestamp.csv";
/********************************/
/* Would you like to add an empty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment integer
/* and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not exist in the table
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file through ftp and
/* change the permissions, or execute at the prompt: touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/


if(!file_exists($csvfile)) {
    echo "File not found. Make sure you specified the correct path.\n";
    exit;
}

$file = fopen($csvfile,"r");

if(!$file) {
    echo "Error opening data file.\n";
    exit;
}

$size = filesize($csvfile);

if(!$size) {
    echo "File is empty.\n";
    exit;
}

$csvcontent = fread($file,$size);

fclose($file);

$con = @mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error());
@mysql_select_db($databasename) or die(mysql_error());

$lines = 0;
$queries = "";
$linearray = array();

foreach(split($lineseparator,$csvcontent) as $line) {

    $lines++;

    $line = trim($line," \t");

    $line = str_replace("\r","",$line);

    /************************************
    This line escapes the special character. remove it if entries are already escaped in the csv file
    ************************************/
    $line = str_replace("'","\'",$line);
    /*************************************/

    $linearray = explode($fieldseparator,$line);

    $linemysql = implode("','",$linearray);

    if($addauto)
        $query = "insert into $databasetable values('','$linemysql');";
    else
        $query = "insert into $databasetable values('$linemysql');";

    $queries .= $query . "\n";

    @mysql_query($query);
}

@mysql_close($con);

if($save) {

    if(!is_writable($outputfile)) {
        echo "File is not writable, check permissions.\n";
    }

    else {
        $file2 = fopen($outputfile,"w");

        if(!$file2) {
            echo "Error writing to the output file.\n";
        }
        else {
            fwrite($file2,$queries);
            fclose($file2);
        }
    }

}

echo "Found a total of $lines records in this csv file.\n All records imported";


?>

---

Answer 1

I thought I would just post the code that solved my issue; it downloads a csv file from an external FTP site onto my local server. It then imports the csv file into my local mysql database table. Once imported the file on the ftp server is deleted. I schedule this job in cpanel with cronjob.

I cannot take credit for the script, I have just modified a script found on another site. I'm sure there are better ways of doing this with some error checking. However, this is the best I could do with my limited skills.

I hope this assists future visitors with the same issue. Thanks to all those that assisted me. Here is the code I used:

<?php

$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server.com") or die("Could not connect");

ftp_login($conn,"ftpusername","ftppassword");
ftp_fget($conn,$target,$source,FTP_ASCII);

echo "file downloaded.\n";

/********************************/
/* Code at http://legend.ws/blog/tips-tricks/csv-php-mysql-import/
/* Edit the entries below to reflect the appropriate values
/********************************/
$dbhost = "localhost";
$dbname = "dbname";
$dbtable = "despgoods";
$dbusername ="dbusername";
$dbpassword = "dbpassword";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "DespGoods.csv";
/********************************/
/* Would you like to add an ampty field at the beginning of these records?
/* This is useful if you have a table with the first field being an auto_increment 
/* integer and the csv file does not have such as empty field before the records.
/* Set 1 for yes and 0 for no. ATTENTION: don't set to 1 if you are not sure.
/* This can dump data in the wrong fields if this extra field does not 
/* exist in the table.
/********************************/
$addauto = 0;
/********************************/
/* Would you like to save the mysql queries in a file? If yes set $save to 1.
/* Permission on the file should be set to 777. Either upload a sample file 
/* through ftp and change the permissions, or execute at the prompt: 
/*     touch output.sql && chmod 777 output.sql
/********************************/
$save = 0;
$outputfile = "output.sql";
/********************************/

if(!file_exists($csvfile)) {
    echo "File not found. Make sure you specified the correct path.\n";
    exit;
}

$file = fopen($csvfile,"r");
if(!$file) {
    echo "Error opening data file.\n";
    exit;
}

$size = filesize($csvfile);
if(!$size) {
    echo "File is empty.\n";
    exit;
}

$csvcontent = fread($file,$size);
fclose($file);

$con = @mysql_connect($dbhost,$dbusername,$dbpassword) or die(mysql_error());
@mysql_select_db($dbname) or die(mysql_error());

$lines = 0;
$queries = "";
$linearray = array();

foreach(split($lineseparator,$csvcontent) as $line) {
    $lines++;
    $line = trim($line," \t");
    $line = str_replace("\r","",$line);

    /************************************
    /* This line escapes the special character. 
    /* Remove it if entries are already escaped in the csv file
    /************************************/
    $line = str_replace("'","\'",$line);
    /*************************************/

    $linearray = explode($fieldseparator,$line);
    $linemysql = implode("','",$linearray);

    if($addauto) {
        $query = "insert into $dbtable values('','$linemysql');";
    }
    else {
        $query = "insert into $dbtable values('$linemysql');";
    }
    $queries .= $query . "\n";
    @mysql_query($query);
}

@mysql_close($con);
if($save) {
    if(!is_writable($outputfile)) {
        echo "File is not writable, check permissions.\n";
    }
    else {
        $file2 = fopen($outputfile,"w");

        if(!$file2) {
            echo "Error writing to the output file.\n";
        }
        else {
            fwrite($file2,$queries);
            fclose($file2);
        }
    }
}

echo "Found a total of $lines records in this csv file.\n";

$source = "DespGoods.csv";
$target = fopen("DespGoods.csv", "w");
$conn = ftp_connect("ftp.server") or die("Could not connect");

ftp_login($conn,"ftpusername","ftppassword");
ftp_delete($conn,$source);
ftp_close($conn);

echo "file deleted";
mysql_close($con);

?>