How to use rsync instead of move in for loop in batch

Question

In every subfolder in the path U:\0012* I create an example folder and move the files from appropriate subfolder to example folder.

FOR /d %%A IN (U:\0012\*) DO mkdir %%~A\example & move %%~A\*.* %%~A\example\ 

What I want to try is to use rsync command instead of move

 FOR /d %%A IN (U:\0012\*) DO 
C:\cygwin\bin\rsync.exe -av -h --progress --checksum "/cygdrive/U/0012/%%A/*.*/" "/cygdrive/U/0012/%%A/EXAMPLE/" 

The script is closing itself with a syntax error. Is it possible to use rsync in for loop?

Answer 1

I think you are having a clash of "Windows World" and "Unix World".

What you are trying with rsync, which is natural Unix tool, is to copy using a batch file path and put it into rsync path. This will not work.

I'll dissect your code:

In this part you are still in the "Windows world" you are using FOR to loop via the U:\0012\* path in a windows way FOR /d %%A IN (U:\0012\*) DO.

You have tried to "help" rsync with unix style path /cygdrive/U/0012/, but the variable %%A contains the windows style path!

C:\cygwin\bin\rsync.exe -av -h --progress --checksum "/cygdrive/U/0012/%%A/*.*/" "/cygdrive/U/0012/%%A/EXAMPLE/"

The error you are getting is probably something like:

The source and destination cannot both be remote. rsync error: syntax or usage error (code 1) at main.c(1292) [Receiver=3.1.2]

How to solve such an error?

First is to say I'm using MSYS environment for linux tools. I have tested it on my local files.

In batch files you could do it the following way:

@echo off
FOR /d %%A IN (U:\0012\*) DO (
    SET "copy_path=%%A"
    C:\msys64\usr\bin\cygpath.exe -u %copy_path% > tmp_file.tmp
    SET /P unix_path=<tmp_file.tmp
    DEL tmp_file.tmp
    ECHO "%unix_path%"
    C:\msys64\usr\bin\rsync.exe -av -h --progress --checksum --remove-source-files "%unix_path%" "%unix_path%/EXAMPLE/"
    )
)

The script takes the Window path C:\prg\PowerShell\test\SO\test1\* and then it uses a cygpath.exe to convert the path to a unix style path and saves it to temporary file (tmp_file.tmp). Then it reads from temporary file the path into the unix variable %unix_path% which is then used in the rsync.exe.

Note: As somebody already linked how to move file with rsync. I'll just state the option. You have to add --remove-source-files, which deletes files after transfer.

** Edit **

• I think I now understand what you mean. Now rsync should really behave like move. Only difference is that if directory example exists it is copied with the file. (if it does not it is created and files moved there) - I think that was original intentions when looking at mkdir && move.

What I have done?

ENABLEDELAYEDEXPANSION to correctly read the variables from file (before it was only the last read)

"!unix_path!/" the last backslash is really important for rsync to behave correctly.

I have tested the script on my computer and for me it works as it should.

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

FOR /D %%A IN (C:\prg\t\*) DO (
    ECHO %%A
    SET "copy_path=%%A"
    C:\msys64\usr\bin\cygpath.exe -u !copy_path! > tmp_file.tmp
    SET /P unix_path=<tmp_file.tmp
    DEL tmp_file.tmp
    ECHO "!unix_path!"

    ECHO C:\msys64\usr\bin\rsync.exe -av --progress --checksum --remove-source-files "!unix_path!/" "!unix_path!/example/"
    C:\msys64\usr\bin\rsync.exe -av --progress --checksum --remove-source-files "!unix_path!/" "!unix_path!/example/"
)

** Second Edit ** Empty directories are not deleted with rsync

I do not recommend using --remove-files && --delete options during one operation due to the fact the rsync can fail. You can read more here superuser.com or unix stack exchange or even server fault.

I would simply go with already written solution: How to delete empty folders using windows command prompt? (I'm talking the solution (all credits to author) that takes in also directories with spaces):

for /f "usebackq delims=" %%d in (`"dir /ad/b/s | sort /R"`) do rd "%%d"

The final script would then look like this:

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

FOR /D %%A IN (C:\prg\t\*) DO (
    ECHO %%A
    SET "copy_path=%%A"
    C:\msys64\usr\bin\cygpath.exe -u !copy_path! > tmp_file.tmp
    SET /P unix_path=<tmp_file.tmp
    DEL tmp_file.tmp
    ECHO "!unix_path!"

    ECHO C:\msys64\usr\bin\rsync.exe -av --progress --checksum --remove-source-files "!unix_path!/" "!unix_path!/example/"
    C:\msys64\usr\bin\rsync.exe -av --progress --checksum --remove-source-files "!unix_path!/" "!unix_path!/example/"
)
REM To delete empty directories - the windows way
REM save current directory to stack
pushd 

cd C:\prg\t
REM Will delete all empty directories (subdirectories) at current path
for /f "usebackq delims=" %%d in (`"dir /ad/b/s | sort /R"`) do rd "%%d"

REM return to saved directory
popd