Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pyspark AttributeError: 'DataFrame' object has no attribute 'toDF'

The following code worked for me before, but not anymore. I got the error:

AttributeError: 'DataFrame' object has no attribute 'toDF'

if __name__ == "__main__":
  sc = SparkContext(appName="test")
  sqlContext = SQLContext(sc)

  df = sqlContext.read.format('com.databricks.spark.csv').\
    options(header='false',delimiter=',',inferSchema='true').load('test')

  ### rename columns
  df = df.toDF('a','b','c')
  ...
  sc.stop()
like image 379
user3610141 Avatar asked Mar 31 '26 01:03

user3610141


1 Answers

I figured it out. Looks like it has to do with our spark version. It worked with 1.6

like image 130
user3610141 Avatar answered Apr 02 '26 21:04

user3610141



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!