Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

commenting in spark sql

How can I comment multiple lines in spark-sql

%sql

/*
    select * from database.tablename 
    where condition
*/

gives me the following error

Error in SQL statement: 

    ParseException: 
    mismatched input '/' expecting {'(', 'CONVERT', 'COPY', 'OPTIMIZE', 'ADD', 'ALTER', 'ANALYZE', 'CACHE', 'CLEAR', 'COMMENT', 'COMMIT', 'CREATE', 'DELETE', 'DESC', 'DESCRIBE', 'DFS', 'DROP', 'EXPLAIN', 'EXPORT', 'FROM', 'GRANT', 'IMPORT', 'INSERT', 'LIST', 'LOAD', 'LOCK', 'MAP', 'MERGE', 'MSCK', 'REDUCE', 'REFRESH', 'REPLACE', 'RESET', 'REVOKE', 'ROLLBACK', 'SELECT', 'SET', 'SHOW', 'START', 'TABLE', 'TRUNCATE', 'UNCACHE', 'UNLOCK', 'UPDATE', 'USE', 'VALUES', 'WITH'}(line 1, pos 0)
like image 211
u6765 Avatar asked Sep 03 '25 16:09

u6765


1 Answers

For single line comment we should use -- and for multiline /* comments */.

Actually comment is working in your case, problem is - spark ignores those comments after that it looks for sql commands but you didn't specify any.

screenshot from databricks

screenshot from databricks

Below code will throw error.

  spark.sql(
    """
      | /* comment..1
      | comment..2 */
      |""")
    .show()

but this works perfectly, since spark got the actual sql command to process.

  spark.sql(
    """
      | /* comment..1
      | comment..2 */
      | select current_date
      |""")
    .show()
like image 125
Mohana B C Avatar answered Sep 05 '25 08:09

Mohana B C