Menu
Do we know any syntax reference for spark-sql? I mean the style like MySQL SELECT syntax, or PostgreSQL Documentation: SELECT.
If not, what is the best source we can access?
473
SparkSql follows Hive style, so you can refer to Hive Syntax for better documentation. The supported and unsupported Hive features by SparkSql can be found in the official documentation.
Use ` to escape special characters (e.g., ` ).
Spark SQL is Apache Spark's module for working with structured data. The SQL Syntax section describes the SQL syntax in detail along with usage examples when applicable. This document provides a list of Data Definition and Data Manipulation Statements, as well as Data Retrieval and Auxiliary Statements.
Go to the SQL tab and find the query you ran. Click on the description to view the visualization of the Spark Directed Acyclic Graph (DAG) for your query execution. You need to read this from top to bottom. You can expand the details at the bottom of the page to view the execution plan for your query.
SparkSql follows Hive style, so you can refer to Hive Syntax for better documentation.
The supported and unsupported Hive features by SparkSql can be found in the official documentation.
83
If not, what is the best source we can access?
I don't know about any better source that could be considered "the best" but the source code of Apache Spark itself. I can imagine that is not as user-friendly as the mentioned sources, but Apache Spark is a highly dynamic open source project and it'd be very hard to keep up with all the changes.
If you're asking about what's supported in so-called SQL mode, you should consult SqlBase.g4 which is the ANTLR grammar of Spark SQL. Unless there's a bug in ANTLR you cannot do anything but what's in the grammar.
35
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
