Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Querying Snowflake via ODBC (using PDO) in PHP returns incorrect data

I'm trying to connect to Snowflake via their ODBC driver in PHP (CentOS 6 and 7) and utilize PDO for easier access.

ODBC driver seems to be installed and working properly, as the native ODBC functions in PHP work pretty well:

$dsn = "Driver=SnowflakeDSIIDriver;Server=" . SNOWFLAKE_HOST;
$dsn .= ";Account=" . SNOWFLAKE_ACCOUNT;
$dsn .= ";Port=" . SNOWFLAKE_PORT;
$dsn .= ";Schema=" . SNOWFLAKE_SCHEMA;
$dsn .= ";Warehouse=" . SNOWFLAKE_WAREHOUSE;
$dsn .= ";Database=" . SNOWFLAKE_DATABASAE;

$conn_id = odbc_connect($dsn, SNOWFLAKE_USER, SNOWFLAKE_PASSWORD);
odbc_exec($conn_id, "USE WAREHOUSE " . SNOWFLAKE_WAREHOUSE);

$res = odbc_exec($conn_id, 'SHOW TABLES IN SCHEMA ' . SNOWFLAKE_SCHEMA . ';');
if ($res) {
    print "Tables in schema\n";
    while($row = odbc_fetch_array($res)) {
        print_r($row);
    }
}

$res = odbc_exec($conn_id, 'SELECT * FROM TEST;');
if ($res) {
    print "Test table content\n";
    while($row = odbc_fetch_array($res)) {
        print_r($row);
    }
}

returns

Tables in schema
Array
(
    [created_on] => 2015-09-09 17:34:43.517000
    [name] => TEST
    [database_name] => TESTSUITE
    [schema_name] => TESTSUITE
    [kind] => TRANSIENT
    [comment] =>
    [cluster_by] =>
    [rows] => 3
    [bytes] => 8192
    [owner] => TESTSUITE
    [account_name] => ****
    [retention_time] => 1
)
Test table content
Array
(
    [C1] => c
    [C2] =>
)
Array
(
    [C1] => a
    [C2] =>
)
Array
(
    [C1] => a
    [C2] =>
)

and that is exactly what I get when I query the DB directly.

But when I want to use PDO, the results are getting weird.

$dsn = "Driver=SnowflakeDSIIDriver;Server=" . SNOWFLAKE_HOST;
$dsn .= ";Account=" . SNOWFLAKE_ACCOUNT;
$dsn .= ";Port=" . SNOWFLAKE_PORT;
$dsn .= ";Schema=" . SNOWFLAKE_SCHEMA;
$dsn .= ";Database=" . SNOWFLAKE_DATABASE;
$dsn .= ";Warehouse=" . SNOWFLAKE_WAREHOUSE;

$pdo = new PDO("odbc:" . $dsn, SNOWFLAKE_USER, SNOWFLAKE_PASSWORD, array(PDO::ATTR_CURSOR => PDO::CURSOR_SCROLL));
$pdo->exec("USE WAREHOUSE " . SNOWFLAKE_WAREHOUSE);

$query = 'SHOW TABLES IN SCHEMA ' . SNOWFLAKE_SCHEMA . ';';
$statement = $pdo->query($query);
print "Tables in schema\n";
print "Rows: " . $statement->rowCount() . "\n";
while ($row = $statement->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT)) {
    print_r($row);
}

$query = 'SELECT * FROM TEST;';
$statement = $pdo->prepare($query);
$statement->execute();
print "Test table content\n";
print "Rows: " . $statement->rowCount() . "\n";
while ($row = $statement->fetch(PDO::FETCH_ASSOC, PDO::FETCH_ORI_NEXT)) {
    print_r($row);
}

returns

Tables in schema
Rows: 1
Array
(
    [created_on] => 2015-09-09 17:34:43.517000
    [name] =>
    [database_name] =>
    [schema_name] =>
    [kind] =>
    [comment] =>
    [cluster_by] =>
    [rows] =>
    [bytes] =>
    [owner] =>
    [account_name] =>
    [retention_time] =>
)
Test table content
Rows: 3

Note: there is not further output after Rows: 3.

So PDOStatement knows the correct number of rows, but the content of SHOW TABLES is incomplete and SELECT * FROM TEST is somehow missing completely ($statement->fetch() returns false immediately).

Any idea if any PDO options would help here?

I have also noticed, that native ODBC functions consume ~200MB of memory in this case, which seems a lot.

like image 929
Ondřej Hlaváček Avatar asked Nov 10 '22 05:11

Ondřej Hlaváček


1 Answers

We had a similar problem, where CAST did not work correctly through PDO with Snowflake.

exception 'PDOException' with message 'SQLSTATE[SL009]: <>: 0 [unixODBC][Driver Manager]No columns were bound prior to calling SQLFetch or SQLFetchScroll

Long story short, after exchanging a couple of e-mails with Snowflake support, we found out the PDO is not supported by Snowflake.

At this time, we do not support PHP PDO resulting from some issues found due to the binding of STRING/VARCHAR type and columns size being 16M bytes. It was a decision to not move forward with supportability. You can continue to use PHP with ODBC, however, we may not be able to address issues/bugs as they arise.

So your best bet is to use native odbc_* family of functions from PHP (this is what we're doing).

like image 69
Grega Kešpret Avatar answered Nov 14 '22 21:11

Grega Kešpret