Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to move files of same extension in databricks files system?

Tags:

databricks

I am facing file not found exception when i am trying to move the file with * in DBFS. Here both source and destination directories are in DBFS. I have the source file named "test_sample.csv" available in dbfs directory and i am using the command like below from notebook cell,

dbutils.fs.mv("dbfs:/usr/krishna/sample/test*.csv", "dbfs:/user/abc/Test/Test.csv")

Error:

java.io.FileNotFoundException: dbfs:/usr/krishna/sample/test*.csv

I appreciate any help. Thanks.

like image 481
Krishna Reddy Avatar asked Jun 08 '18 13:06

Krishna Reddy


1 Answers

Wildcards are currently not supported with dbutils. You can move the whole directory:

dbutils.fs.mv("dbfs:/tmp/test", "dbfs:/tmp/test2", recurse=True)

or just a single file:

dbutils.fs.mv("dbfs:/tmp/test/test.csv", "dbfs:/tmp/test2/test2.csv")

As mentioned in the comments below, you can use python to implement this wildcard-logic. See also some code examples in my following answer.

like image 128
Hauke Mallow Avatar answered Oct 08 '22 19:10

Hauke Mallow