Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

New posts in apache-spark

How to read from textfile(String type data) map and load data into parquet format(multiple columns with different datatype) in Spark scala dynamically

Unable to get S3A Directory Committers to write files in Spark 3.0.0

apache-spark amazon-s3

Aggregate data from different micro batches in Spark streaming

How to unzip the files stored in hdfs using spark java

java hadoop apache-spark hdfs

Upload Spark RDD to REST webservice POST method

Scala Patch function

scala apache-spark

How to change the schema of a DataFrame (to fix the names of some nested fields)?

Pyspark - from_unixtime not showing the correct datetime

How to add a new column to a Delta Lake table?

explode a row of spark dataset into several rows with added column using flatmap

Spark fails to merge parquet files (INTEGER -> DECIMAL)

Spark SQL - How do i set a variable within the query, to re-use throughout?

Spark - Csv data split with scala

scala csv apache-spark