How to add empty map type column to DataFrame?

Question

I want to add a new map type column to a dataframe, like this:

|-- cMap: map (nullable = true)
|    |-- key: string
|    |-- value: string (valueContainsNull = true)

I tried the code:

df.withColumn("cMap", lit(null).cast(MapType)).printSchema

The error is :

<console>:132: error: overloaded method value cast with alternatives:
(to: String)org.apache.spark.sql.Column <and>
(to: org.apache.spark.sql.types.DataType)org.apache.spark.sql.Column
cannot be applied to (org.apache.spark.sql.types.MapType.type)

Is there other way to cast the new column to Map or MapType? Thanks

Answer 1

I had the same problem, finally I found solution:

df.withColumn("cMap", typedLit(Map.empty[String, String])) 

From ScalaDocs for typedLit:

The difference between this function and [[lit]] is that this function can handle parameterized scala types e.g.: List, Seq and Map.