Spark Version in Databricks

Question

How do I determine which version of Spark I'm running on Databricks? I would like to try koalas, but when I try import databricks.koalas, it returns a "No module named databricks" error message. When I try from databricks import koalas, it returns the same message.

Answer 1

Koalas is only included into the Databricks Runtime versions 7.x and higher. It's not included into DBR 6.x. You can find version of Databricks Runtime in the UI, if you click on dropdown on top of the notebook.

You can check version of Koalas in the Databricks Runtime release notes.