Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to connect any RDBMS through spark usinig java?

            JdbcRDD rdd = new org.apache.spark.rdd.JdbcRDD(
                    sparkConf,
                    ()= > {
                        Class.forName ("com.mysql.jdbc.Driver")
    sql.DriverManager.getConnection("jdbc:mysql://mysql.example.com/?user=batman&password=alfred")
  },
  "SELECT * FROM BOOKS WHERE ? <= KEY AND KEY <= ?",
  0, 1000, 10,
  row =  > row.getString("BOOK_TITLE")
         )

I tried above scala code changing classes to java 8,but so many errors coming.

like image 707
kunal Avatar asked Aug 20 '14 10:08

kunal


1 Answers

I met the same problem before, it turns out it's the problem of SQL parameters, basically you need to use sql like: sql select * from books limit ?, ? the two parameters for lowerBound and upperBound which are required by JdbcRdd constructor.

like image 52
smilingleo Avatar answered Sep 30 '22 18:09

smilingleo