Glennie Helles Sindholt

Glennie Helles Sindholt has asked 3 questions and find answers to 62 problems.

Stats

About

I have a Ph.D in computer science and my current passion is Big Data and in particular Spark :)

Glennie Helles Sindholt questions

How to control number of parquet files generated when using partitionBy
Is groupByKey ever preferred over reduceByKey
Spark code organization and best practices [closed]

Glennie Helles Sindholt answers

cast schema of a data frame in Spark and Scala
Spark Dataframe distinguish columns with duplicated name
How to access elemens in Row RDD in SCALA
write a spark Dataset to json with all keys in the schema, including null columns
How to load only the data of the last partition
How to upload files to new EMR cluster
spark df.write.partitionBy run very slow
Get all Apache Spark executor logs
Apache Spark: Using folder structures to reduce run-time of analyses
Spark: what's the advantages of having multiple executors per node for a Job?