Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I use rsync to retrieve files from an HTTP URL?

There is a bunch of files here: http://data.pudo.org/flexicadastre/

I want to have a local folder that contains the same files there are in that URL.

I've tried:

rsync -av http://data.pudo.org/flexicadastre/ .

But I get:

ssh: Could not resolve hostname http: nodename nor servname provided, or not known
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [receiver=2.6.9]

I normally use rsync to synchronise local folders or folder via FTP or SSH, but this is different and I'm not sure how to handle it. This site looks like an FTP to me, but it's exposed via HTTP. I can't really understand how it works. May rsync is not possible in this scenario and I should use something else?

like image 455
zapatilla Avatar asked Nov 17 '15 11:11

zapatilla


People also ask

How do I retrieve files from rsync?

If you want to copy a file from one location to another within your system, you can do so by typing rsync followed by the source file name and the destination directory. Note: Instead of “/home/tin/file1. txt”, we can also type “file1” as we are currently working in the home directory.

Does rsync use SSH?

Here's how to transfer a file from a remote server to your local system, also called a “pull” operation. Note: When working with remote systems, make sure you have SSH access to the remote system. Rsync establishes the connection using SSH in order to enable file transfer.

How do I use rsync with FTP?

Short answer – You can't. rsync can't use ftp as a remote host. So rsync does not work over ftp session/protocol.


1 Answers

It is not possible to use rsync on http URLs.

To use the rsync algorithm the client-side rsync needs to interact with a server-side rsync process. This is done either directly, through ssh or (less common) rsh. HTTP is not an option.

May rsync is not possible in this scenario and I should use something else?

It looks like that you only want to mirror data from HTTP to the local file system. There are several programs out there which does this like wget or lwp-mirror and probably lots of others.

like image 118
Steffen Ullrich Avatar answered Sep 29 '22 05:09

Steffen Ullrich