Is there a way to set a name to a csv file in sparklyr using spark_write_csv?

Question

I need to write a data frame to a single csv file, and found out that I can use sdf_coalesce() to turn the file into a single partition. I want to find out if there's any way I can change the name of the csv file generated by spark_write_csv()?

Thanks in advance.

Answer 1

No. Filename is generated automatically to be unique among different tasks and it is not configurable. If you want specific name you have rename the output afterwards using utilities specific for the file system / storage solution in use.