Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How to check Spark Version [closed]

Tags:

apache-spark

hadoop

cloudera

I want to check the spark version in cdh 5.7.0. I have searched on the internet but not able to understand. Please help.

like image 422
Ironman Avatar asked Jul 26 '16 10:07

Ironman

People also ask

How do I check my spark version?

Click on Admin -> Stack and Versions and you will find the version information under Version tab.

How do I open spark in terminal?

Go to the Apache Spark Installation directory from the command line and type bin/spark-shell and press enter, this launches Spark shell and gives you a scala prompt to interact with Spark in scala language. If you have set the Spark in a PATH then just enter spark-shell in command line or terminal (mac users).

1 Answers

Addition to @Binary Nerd

If you are using Spark, use the following to get the Spark version:

spark-submit --version

or

Login to the Cloudera Manager and goto Hosts page then run inspect hosts in cluster

like image 108
BruceWayne Avatar answered Sep 24 '22 19:09

BruceWayne
Sign in to Comment

Related questions

Why does Spark fail with java.lang.OutOfMemoryError: GC overhead limit exceeded?
Difference between == and === in Scala, Spark
'PipelinedRDD' object has no attribute 'toDF' in PySpark
Pyspark: Pass multiple columns in UDF
Importing spark.implicits._ in scala
Which operations preserve RDD order?
Why does a job fail with "No space left on device", but df says otherwise?
What is the difference between Apache Mahout and Apache Spark's MLlib?
PySpark groupByKey returning pyspark.resultiterable.ResultIterable
Median / quantiles within PySpark groupBy
Upacking a list to select multiple columns from a spark data frame
Apache Spark -- Assign the result of UDF to multiple dataframe columns
PySpark: withColumn() with two conditions and three outcomes
How to flatten a struct in a Spark dataframe?
Automatically and Elegantly flatten DataFrame in Spark SQL
How to split Vector into columns - using PySpark
aggregate function Count usage with groupBy in Spark
What are the various join types in Spark?
How does Spark partition(ing) work on files in HDFS?
How to melt Spark DataFrame?

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!