Dennis Huo

Dennis Huo has asked 3 questions and find answers to 47 problems.

Stats

About

Software Engineer at Google working on Big Data Open-Source Software on Google Cloud Platform. Tech lead of Google Cloud Dataproc, Google's managed Hadoop and Spark service.

Dennis Huo questions

How can I run uncompiled Spark Scala/spark-shell code as a Dataproc job?
Getting NullPointerException using spark-csv with DataFrames

Dennis Huo answers

Creating a cluster before sending a job to dataproc programmatically
Can't add jars pyspark in jupyter of Google DataProc
How to launch Spark's ApplicationMaster on a particular node in YARN cluster?
How do I use gcloud to find which regions my dataproc clusters are in?
Run Bash script on GCP Dataproc
Spark Cluster, failed to connect to master. (WARN Worker: Failed to connect to master)
How do I get a spark job to use all available resources on a Google Cloud DataProc cluster?
Why is rdd.map(identity).cache slow when rdd items are big?
Spark SQL - loading csv/psv files with some malformed records
"No Filesystem for Scheme: gs" when running spark job locally