In JDBC To Other Databases I found the following explanation of dbtable
parameter:
The JDBC table that should be read. Note that anything that is valid in a FROM clause of a SQL query can be used. For example, instead of a full table you could also use a subquery in parentheses.
When I use the code:
CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
url "jdbc:postgresql:dbserver",
dbtable "mytable"
)
everything works great, but the following:
dbtable "SELECT * FROM mytable"
leads to the error:
What is wrong?
Since dbtable
is used as a source for the SELECT
statement it has be in a form which would be valid for normal SQL query. If you want to use subquery you should pass a query in parentheses and provide an alias:
CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
url "jdbc:postgresql:dbserver",
dbtable "(SELECT * FROM mytable) tmp"
);
It will be passed to the database as:
SELECT * FROM (SELECT * FROM mytable) tmp WHERE 1=0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With