scp and remote mkdir -p

Question

hi i have some file path like /ifshk5/BC_IP/PROJECT/T1 1073/T11073_RICljiR/split/AG19_235/120225_I872_FCC0HN2ACXX_L8_RICljiRSYHSD2-1-IP AAPEK-17_1.fq.gz

i need copy files from one ftp server to other. and also need to create directory if it not exist in server. i login the sever which contains those file then run this code

 #! /bin/bash

 while read myline
 do
   for i in $myline
    do
    if [ -f $i ]
    then
    location=$(echo "$i" | awk -F "/" '{ print "", $6, $7, $8 }' OFS="/")
        #location shows /T11073_RICekkR/Fq/AS59_59304
    location="/opt/CLiMB/Storage3/ftp/ftp_climb/100033"$location
    echo $location

    ssh [email protected] mkdir -p $location
    scp -r $i [email protected]:$location

    fi
   done
 done < /ifshk5/BC_IP/PROJECT/T11073/T11073_all_3254.fq.list

it has some problem, 1. it can't work always shows permission denied, please try again. but when i direct type

 ssh [email protected] mkdir -p /sample/xxxx

it can work, and the new dir location is right it shows like /opt/CLiMB/Storage3/ftp/ftp_climb/100033/T11073_RICekkR/Fq/AS59_59304

Answer 1

I don't see where the "permission denied" error might come from; run the script with bash -x to see the command which causes the error. Maybe it's not what you expect.

Also try rsync instead of inventing the wheel again:

rsync --dirs $i [email protected]:$b

--dirs will create the necessary folders on the remote side (and it will give you good error messages when something fails).

It might even be possible to do everything with a single call to rsync if you have the same folder structure on both sides:

rsync -avP /ifshk5/BC_IP/PROJECT/T11073/ [email protected]:/opt/CLiMB/Storage3/ftp/ftp_climb/100033/

Note the / after the paths! Don't omit them.

rsync will figure out which files need to be transferred and copy only those. If you want to transfer only a subset, use --include-from