Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

New posts in apache-spark

PySpark / Spark Window Function First/ Last Issue

What are the differences between slices and partitions of RDDs?

python apache-spark

How to convert a case-class-based RDD into a DataFrame?

Creating a new Spark DataFrame with new column value based on column in first dataframe Java

How to convert column values from string to decimal?

Is there a way to get the column data type in pyspark?

apache-spark pyspark

How Can I Obtain an Element Position in Spark's RDD?

position apache-spark rdd

Read Array of String from Spark

scala apache-spark

Spark SQL: How to append new row to dataframe table (from another table)

How to save a partitioned parquet file in Spark 2.1?

How do I delete files in hdfs directory after reading it using scala?

File already exists error writing new files from dataframe

apache-spark emr

Kafka Structured Streaming KafkaSourceProvider could not be instantiated

How to get rid of "Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties" message?

log4j apache-spark

Is there a way to filter a field not containing something in a spark dataframe using scala?

Spark SQL change format of the number

key not found: _PYSPARK_DRIVER_CALLBACK_HOST

python apache-spark pyspark

Error while using Hive context in spark : object hive is not a member of package org.apache.spark.sql

Scala/Spark version compatibility

scala apache-spark

Selecting only numeric/string columns names from a Spark DF in pyspark