scp fails with "protocol error: filename does not match request"

Question

I have a script that uses SCP to pull a file from a remote Linux host on AWS. After running the same code nightly for about 6 months without issue, it started failing today with protocol error: filename does not match request. I reproduced the issue on some simpler filenames below:

$ scp -i $IDENT $HOST_AND_DIR/"foobar" .
# the file is copied successfully

$ scp -i $IDENT $HOST_AND_DIR/"'foobar'" .
protocol error: filename does not match request
# used to work, i swear...

$ scp -i $IDENT $HOST_AND_DIR/"'foobarbaz'" .
scp: /home/user_redacted/foobarbaz: No such file or directory
# less surprising...

The reason for my single quotes was that I was grabbing a file with spaces in the name originally. To deal with the spaces, I had done $HOST_AND_DIR/"'foo bar'" for many months, but starting today, it would only accept $HOST_AND_DIR/"foo\ bar". So, my issue is fixed, but I'm still curious about what's going on.

I Googled the error message, but I don't see any real mentions of it, which surprises me.

Both hosts involved have OpenSSL 1.0.2g in the output of ssh -v localhost, and bash --version says GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu) Any ideas?

Answer 1

I ended up having a look through the source code and found the commit where this error is thrown:

GitHub Commit

remote->local directory copies satisfy the wildcard specified by the user.

This checking provides some protection against a malicious server sending unexpected filenames, but it comes at a risk of rejecting wanted files due to differences between client and server wildcard expansion rules.

For this reason, this also adds a new -T flag to disable the check.

They have added a new flag -T that will ignore this new check they've added so it is backwards compatible. However, I suppose we should look and find out why the filenames we're using are flagged as restricted.

Answer 2

In my case, I had [] characters in the filename that needed to be escaped using one of the options listed here. for example:

scp USERNAME@IP_ADDR:"/tmp/foo\[bar\].txt" /tmp