How to access SparkContext from SparkSession instance?

Question

I am importing SparkSession as follows in PySpark:

from pyspark.sql import SparkSession

Then I create SparkSession:

spark = SparkSession.builder.appName("test").getOrCreate()

and try to access SparkContext:

spark.SparkContext.broadcast(...)

However, I get an error that SparkContext does not exist. How can I access it in order to set broadcast variables?

Answer 1

You almost got it right, it's lowercase s at the beginning:

>>> spark.sparkContext
<SparkContext master=local[*] appName=PySparkShell>

Answer 2

Asumming you have a spark session

spark_session = SparkSession \
    .builder \
    .enableHiveSupport() \
    .getOrCreate()

---

Spark Context can be inferred using

spark_context = spark_session._sc

or

spark_context = spark_session.sparkContext