Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use SQL query to define table in dbtable?

In JDBC To Other Databases I found the following explanation of dbtable parameter:

The JDBC table that should be read. Note that anything that is valid in a FROM clause of a SQL query can be used. For example, instead of a full table you could also use a subquery in parentheses.

When I use the code:

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
  url "jdbc:postgresql:dbserver",
  dbtable "mytable"
)

everything works great, but the following:

 dbtable "SELECT * FROM mytable"

leads to the error:

enter image description here

What is wrong?

like image 809
SkyFox Avatar asked Dec 04 '22 02:12

SkyFox


1 Answers

Since dbtable is used as a source for the SELECT statement it has be in a form which would be valid for normal SQL query. If you want to use subquery you should pass a query in parentheses and provide an alias:

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
    url "jdbc:postgresql:dbserver",
    dbtable "(SELECT * FROM mytable) tmp"
);

It will be passed to the database as:

SELECT * FROM (SELECT * FROM mytable) tmp WHERE 1=0
like image 200
zero323 Avatar answered Jan 19 '23 05:01

zero323