I'm trying to convert Row
of DataFrame
into json string using only spark API.
From input Row
+----------------+-----------+
| someThing| else|
+----------------+-----------+
| life| 42|
+----------------+-----------+
with
myDataFrame
.select(struct("*").as("col"))
.select(to_json(col("col")))
.writeStream()
.foreach(new KafkaWriter())
.start()
using KafkaWriter
, that is using row.toString()
i got:
[{
"someThing":"life",
"else":42
}]
When i would like to get this instead:
{
"someThing":"life",
"else":42
}
(without the []
)
Any idea?
Just found the solution. Using Row.mkString
instead of Row.toString
solved my case.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With