The following code worked for me before, but not anymore. I got the error:
AttributeError: 'DataFrame' object has no attribute 'toDF'
if __name__ == "__main__":
sc = SparkContext(appName="test")
sqlContext = SQLContext(sc)
df = sqlContext.read.format('com.databricks.spark.csv').\
options(header='false',delimiter=',',inferSchema='true').load('test')
### rename columns
df = df.toDF('a','b','c')
...
sc.stop()
I figured it out. Looks like it has to do with our spark version. It worked with 1.6
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With