Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

recursively use scp but excluding some folders

Tags:

bash

scp

Assume there are some folders with these structures

/bench1/1cpu/p_0/image/ /bench1/1cpu/p_0/fl_1/ /bench1/1cpu/p_0/fl_1/ /bench1/1cpu/p_0/fl_1/ /bench1/1cpu/p_0/fl_1/ /bench1/1cpu/p_1/image/ /bench1/1cpu/p_1/fl_1/ /bench1/1cpu/p_1/fl_1/ /bench1/1cpu/p_1/fl_1/ /bench1/1cpu/p_1/fl_1/ /bench1/2cpu/p_0/image/ /bench1/2cpu/p_0/fl_1/ /bench1/2cpu/p_0/fl_1/ /bench1/2cpu/p_0/fl_1/ /bench1/2cpu/p_0/fl_1/ /bench1/2cpu/p_1/image/ /bench1/2cpu/p_1/fl_1/ /bench1/2cpu/p_1/fl_1/ /bench1/2cpu/p_1/fl_1/ /bench1/2cpu/p_1/fl_1/ .... 

What I want to do is to scp the following folders

/bench1/1cpu/p_0/image/ /bench1/1cpu/p_1/image/ /bench1/2cpu/p_0/image/ /bench1/2cpu/p_1/image/ 

As you can see I want to recursively use scp but excluding all folders that name "fl_X". It seems that scp has not such option.

UPDATE scp has not such feature. Instead I use the following command

 rsync -av --exclude 'fl_*' user@server:/my/dir 

But it doesn't work. It only transfers the list of folders!! something like ls -R

like image 823
mahmood Avatar asked Feb 27 '13 20:02

mahmood


People also ask

What is difference between SCP and rsync?

Copying files and directories with SCP or Rsync Secure Copy (SCP) uses SSH to copy only the files or directories that you select. On first use, Rsync copies all files and directories and then it copies only the files and directories that you have changed. It does not copy all the files and directories again.

Does SCP work on folders?

The Unix command scp (which stands for "secure copy protocol") is a simple tool for uploading or downloading files (or directories) to/from a remote machine.

How do I SCP all contents of a directory?

To copy a directory (and all the files it contains), use scp with the -r option. This tells scp to recursively copy the source directory and its contents. You'll be prompted for your password on the source system ( deathstar.com ). The command won't work unless you enter the correct password.

How do I exclude a file in Linux?

To do so, create a text file with the name of the files and directories you want to exclude. Then, pass the name of the file to the --exlude-from option.

How do I exclude files when using SCP to copy a directory?

However, there is a great workaround to exclude files and copy it securely using ssh. This page explains how to filter or excludes files when using scp to copy a directory recursively. -a : Recurse into directories i.e. copy all files and subdirectories. Also, turn on archive mode and all other options (-rlptgoD)

How do I copy files from remote to local using SCP?

scp -r.. 192.168.1.20:/mnt/backup Copy Recursively From Remote To The Local The scp command can be used to copy remote files and directories to the local system recursively. The -r option is used copy files and directories recursively.

What is the use of SCP in Linux?

January 19, 2021 by İsmail Baydan The scp is a command and tool used to copy files and directories between two systems over the network. The scp command is generally used for Linux and network systems but also supports Windows operating systems. One of the most useful features of the scp command is copying files and directories recursively.

How to copy files and directories recursively from remote to local?

The scp command can be used to copy remote files and directories to the local system recursively. The -r option is used copy files and directories recursively. The following syntax can be used copy recursively from remote to the local. scp -r REMOTE_PATH LOCAL_PATH . LOCAL_PATH is the local path or directory we want to copy to the remote.


1 Answers

Although scp supports recursive directory copying with the -r option, it does not support filtering of the files. There are several ways to accomplish your task, but I would probably rely on find, xargs, tar, and ssh instead of scp.

find . -type d -wholename '*bench*/image' \ | xargs tar cf - \ | ssh user@remote tar xf - -C /my/dir 

The rsync solution can be made to work, but you are missing some arguments. rsync also needs the r switch to recurse into subdirectories. Also, if you want the same security of scp, you need to do the transfer under ssh. Something like:

rsync -avr -e "ssh -l user" --exclude 'fl_*' ./bench* remote:/my/dir 
like image 197
jxh Avatar answered Sep 28 '22 06:09

jxh