I haven't found them after having a cursory glance at the Spark codebase. In most documentation and tutorial examples, people seem to be using 'naked' string literals to specify join types. Does Spark provide an object or class defining "leftouter", "inner", "cartesian" etc. as public vals, or is relying on string literals simply the convention?
That is to say, is there an alternative to:
dataframe.join(
right = anotherDataFrame,
joinExprs = expr("1 = 1"),
joinType = "leftouter"
)
that would look something like:
dataframe.join(
right = anotherDataFrame,
joinExprs = expr("1 = 1"),
joinType = SparkJoins.LeftOuter
)
?
You could use the the objects included in the package, for instance, using the following objects for each literal:
org.apache.spark.sql.catalyst.plans.Inner.sql
org.apache.spark.sql.catalyst.plans.LeftOuter.sql
source: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/joinTypes.scala
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With