Using wildcards in file names using Python's SCPClient library

Question

I want to copy files from remote machine using Python script to my local machine. I only know the extension of the file names so I want to use wildcards to express the file name.

In addition, I want to use the SCPClient Python library and not the os.system directly as suggested in the question titled using wildcards in filename in scp in python

But when I run the following code:

from paramiko import SSHClient
import paramiko
from scp import SCPClient

with SSHClient() as ssh:
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    ssh.connect('10.10.100.5', username= 'root', password='Secret')
    with SCPClient(ssh.get_transport()) as scp:
        scp.get(remote_path='/root/*.py', local_path='.')

I get an exception

scp.SCPException: scp: /root/*.py: No such file or directory

Running from shell works just fine

scp [email protected]:/root/*.py .

Answer 1

You need to add sanitize to your get_transport():

with SCPClient(ssh.get_transport(), sanitize=lambda x: x) as scp:
        scp.get(remote_path='/root/*.py', local_path='.')

Otherwise wildcards are treated literally.

EDIT Sep 2020: There has been some confusion about this answer. The above code does not sanitise anything but it provides the mandated sanitisation function. Paramiko does not have a default sanitiser we would now be turning off. Whatever sanitisation happens or does not happen must come from the implementation, as there is no way for Paramiko to know which paths are safe in your particular environment.

If there is no sanitising function present at all, all wildcards are interpreted literally - I assume for security reasons as wildcards are dangerous with untrusted input. The design forces the implementation to consider sanitisation by adding a sanitising function of some kind. If there is none, wildcards do not work as wildcards.

To make wildcards work, there needs to be a sanitising function that returns the sanitised path. The above lambda is a dummy function that returns whatever is fed to it - which means it does not do any sanitisation at all but as there now is a sanitising function, wildcards start now working as wildcards.

If you do not trust your inputs, do not do this. In that case you need to write a proper sanitiser that will make sure the path is safe to use. You will need to write a function that returns the sanitised path.