Pyspark StructType is not defined

Question

I'm trying to struct a schema for db testing, and StructType apparently isn't working for some reason. I'm following a tut, and it doesn't import any extra module.

<type 'exceptions.NameError'>, NameError("name 'StructType' is not defined",), <traceback object at 0x2b555f0>) 

I'm on spark 1.4.0, and Ubuntu 12 if that has anything to do with the problem. How would I fix this problem? Thank you in advance.

Answer 1

Did you import StructType? If not

from pyspark.sql.types import StructType 

should solve the problem.

Answer 2

from pyspark.sql.types import StructType 

That would fix it but next you might get NameError: name 'IntegerType' is not defined or NameError: name 'StringType' is not defined ..

To avoid all of that just do:

from pyspark.sql.types import * 

Alternatively import all the types you require one by one:

from pyspark.sql.types import StructType, IntegerType, StringType 

All Types: Apache Spark Data Types