Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I export Scala Spark DataFrames schema to a Json file?

I have tried df.write.json("myNewJson")which although does not throws any error or warning it simply doesn't create any file in the target. df.write.format("json").save("/MyDir/") also seems to have the same problem. Am I missing some statement here?

like image 348
Chetan SP Avatar asked Dec 24 '22 08:12

Chetan SP


1 Answers

Posting the exact code that worked for me for those who might stumble upon the same problem.. I was able to export a DataFrame schema to Json file using the below statements:

import java.io._

val a = df.schema.json
val writer = new PrintWriter(new File("/home/file"))
writer.write(a)
writer.close()
like image 166
Chetan SP Avatar answered Mar 07 '23 13:03

Chetan SP