How to read file through ssh/scp directly

Question

I have a program written in C/C++ that reads two files and then generate some reports. The typical workflow is as follows:

1> scp user@server01:/temp/file1.txt ~/ then input my password for the prompty

2> my_program file1.txt localfile.txt

Is there a way that I can let my program directly handle the remote file without explicitly copying the file to local first? I have tried the following command but it doesn't work for me.

> my_program <(ssh user@server01:/temp/file1.txt) localfile.txt

Answer 1

If you are really on Red Hat Linux, this should work:

ssh user@host cat /temp/file1.txt | my_program /dev/stdin localfile.txt

Using a single - instead of /dev/stdin may work as well, but it would depend on whether my_program was written to interpret an argument of - as "read this file from stdin".

Also, this might not work if my_program expects to be able to seek around in the first file (I/O streams are generally non-seekable), but if all it does is read from the first byte to the last byte, it should be OK.

Answer 2

You need to use ssh user@host "cat /temp/file.txt".

You can use this to redirect it to stdin of your program: ssh user@host "cat /temp/file.txt" | my_program or you can actually call this in your program with an fork/exec. Or like you described it to generate a temporary file: my_program <(ssh user@host "cat /temp/file.txt")

Other options would be to use sshfs with fuse or using a ssh (or VFS) library in your program.