I want to get all the tables names from a sql query in Spark using Scala.
Lets say user sends a SQL query which looks like:
select * from table_1 as a left join table_2 as b on a.id=b.id
I would like to get all tables list like table_1
and table_2
.
Is regex the only option ?
-- Query to Get SQL Server Database Table Names USE [AdventureWorksDW2014] GO SELECT * FROM sys.tables. OUTPUT. You can also select the required columns from the sys.tables using below shown query. By this, you can see the required columns such as Table Name, Created Date, and Table Modified Date etc.
The easiest way to find all tables in SQL is to query the INFORMATION_SCHEMA views. You do this by specifying the information schema, then the “tables” view. Here’s an example. SELECT table_name, table_schema, table_type FROM information_schema.tables ORDER BY table_name ASC; This will show the name of the table, which schema it belongs to, ...
We can verify the data in the table using the SELECT query as below. We will be using sys. columns to get the column names in a table. It is a system table and used for maintaining column information. It contains the following information about columns: Name – Name of the column.
You can query the table such as select query,substring (query,st,st-case when e=0 then 1 else charindex ('from',query) end) Thanks for the reply.
Thanks a lot @Swapnil Chougule for the answer. That inspired me to offer an idiomatic way of collecting all the tables in a structured query.
scala> spark.version
res0: String = 2.3.1
def getTables(query: String): Seq[String] = {
val logicalPlan = spark.sessionState.sqlParser.parsePlan(query)
import org.apache.spark.sql.catalyst.analysis.UnresolvedRelation
logicalPlan.collect { case r: UnresolvedRelation => r.tableName }
}
val query = "select * from table_1 as a left join table_2 as b on a.id=b.id"
scala> getTables(query).foreach(println)
table_1
table_2
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With