Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

php postgresql pdo copy from stdin

COPY table_name ( field1, field2, field3) FROM STDIN CSV;
1,2,"q w"
3,4,"a s"
5,6,d
\.

How to execute this query by PDO ?

Update:

Problem is PDO driver executes this query as statement.
For example, if you paste it into pgAdmin, it throws an error.
I need execute it in psql:

C:\Users\User>psql -e -h localhost -U postgres db_name
psql (9.1.2)
db_name=# COPY table_name ( field1, field2, field3) FROM STDIN CSV;
COPY table_name ( field1, field2, field3) FROM STDIN CSV;
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> 1,2,"q w"
>> 3,4,"a s"
>> 5,6,d
>> \.
like image 643
cetver Avatar asked Feb 13 '12 14:02

cetver


1 Answers

Thanks to this book

Note that the function presented here effectively by-passes security restrictions, which are there for a reason. Your function should check the file path and table provided against strict white list conditions. This example is also open to SQL injection as it does not quote its input correctly.

Create function which execute COPY command

CREATE OR REPLACE FUNCTION copy_from_csv_ignoring_security(table_name text, table_fieds text, file_path text, oids boolean DEFAULT false, header boolean DEFAULT false, delimeter text DEFAULT ','::text, "null" text DEFAULT ''::text, quote text DEFAULT '"'::text, escape text DEFAULT '"'::text, force_not_null text DEFAULT ''::text)
  RETURNS void AS
$BODY$

declare statement text;
begin

statement := 'COPY ' || table_name || ' (' || table_fieds || ') ' || 'FROM ''' || file_path || ''' WITH ';
IF oids THEN
 statement := statement || 'OIDS ';
end if;
statement := statement || 'DELIMITER ''' || delimeter || ''' ';
statement := statement || 'NULL ''' || "null" || ''' CSV ';
IF header THEN
 statement := statement || 'HEADER ';
end if;
statement := statement || 'QUOTE ''' || "quote" || ''' ';
statement := statement || 'ESCAPE ''' || "escape" || ''' ';
IF force_not_null <> '' THEN
statement := statement || 'FORCE NOT NULL ''' || force_not_null || ''' ';
end if;
execute statement;
end;
$BODY$
  LANGUAGE plpgsql VOLATILE SECURITY DEFINER
  COST 100;

Give rights on function

revoke all on function copy_from_csv_ignoring_security(text, text, text, boolean, boolean, text, text, text, text, text) from public;
grant execute on function copy_from_csv_ignoring_security(text, text, text, boolean, boolean, text, text, text, text, text) to db_user;

Execute from PHP

$dbh->exec('SELECT copy_from_csv_ignoring_security(...)');

===== If version >= 9.1.7 trick above doesn't works. =====

Solution:

create file .pgpass (avoid password prompt) in home directory of user which run this script.

#.pgpass contents (chmod 600 - requred)    
host:port:db_name:user_name:password

create php function, which executes meta-command

function executeMetaCommand($dbUser, $dbName, $dbPort, $command)
{    
    $command = sprintf(
        "psql -U %s -p %s -d %s -f - <<EOT\n%s\nEOT\n",
        $dbUser, $dbPort, $dbName, $command
    );
    $streams = array(
        array('pipe', 'r'),// stdin
        array('pipe', 'w'),// stdout
        array('pipe', 'w') // stderr
    );
    $process = proc_open($command, $streams, $pipes);
    if (!is_resource($process)) {
        throw new Exception("Cannot open process:\n$command");
    } else {
        list(, $stdout, $stderr) = $pipes;
        $error = stream_get_contents($stderr);
        fclose($stderr);
        if (strlen($error) > 0) {
            throw new Exception("Process error:\n$error");
        } else {
            $output = stream_get_contents($stdout);
            fclose($stdout);
            $returnCode = proc_close($process);
            if ($returnCode === -1) {
                throw new Exception("Process was completed incorrectly:\n$output");
            } else {
                return array(
                    $returnCode,
                    $output
                );
            }
        }
    }
}

//usage:
$command = sprintf("\\copy table(field1, field2) FROM '%s' WITH CSV", $filePath);
executeMetaCommand('postgres', 'test_db', '5432', $command);
like image 119
cetver Avatar answered Nov 12 '22 04:11

cetver