Cannot access temp table created by createOrReplaceGlobalTempView

Question

My code is not working in emr

finalDF.createOrReplaceGlobalTempView("temp_visits")
spark.sql(s"insert overwrite table test PARTITION (date) SELECT * from temp_visits")

I am getting the following exception

Exception in thread "main" org.apache.spark.sql.AnalysisException: Table or view not found: temp_visits; line 1 pos 100
    at org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
    at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.org$apache$spark$sql$catalyst$analysis$Analyzer$ResolveRelations$$lookupTableFromCatalog(Analyzer.scala:663)

Is there any setting I need to add for this temp table to be recognised?

Answer 1

need to add database name in the referenced global view.
Default spark settings have database name - global_temp

finalDF.createOrReplaceGlobalTempView("temp_visits")
spark.sql(s"insert overwrite table test PARTITION (date) SELECT * from global_temp.temp_visits")

you can change this name by providing configuration during sparkSession initialisation -

spark.sql.globalTempDatabase