Has been discussed that the way to find the column datatype in pyspark is using df.dtypes get datatype of column using pyspark. The problem with this is that for datatypes like an array or struct you get something like array<string>
or array<integer>
.
Question: Is there a native way to get the pyspark data type? Like ArrayType(StringType,true)
Just use schema
:
df.schema[column_name].dataType
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With