Auto Sync Azure Blob to local file system [closed]

Question

I would like to know if there is a way through which I can auto-sync an Azure blob to the local file system, so that whenever a blob gets added to a container an event is fired so it can be downloaded to a local folder.

So far I can sync from the local file system to Azure blobs, but not the other way.

If I use polling, how frequently should I poll and how much does it affect performance?

Answer 1

This is something you'd need to manage yourself, as there's no way to watch a blob or container. That said: you can check a container's (or blob's) eTag to see if content has been updated.

As for polling: each time you do a GET, you'll incur a transaction hit. Not that it's very costly (a half-penny per 100K transactions): If you polled once every second, you'd spent maybe 15 cents (per role instance doing the polling) monthly. However: I think that's a bit aggressive, especially if you have other storage-related activities happening simultaneously (queue-polling, etc.).

This comes down to how often you think blob content will be updated, and how current your local cache needs to be.

You can also consider building an uploader service that takes an uploaded object, stores it in blob storage, and notifies all running instances that there's an update (including URL to the blob updated). Maybe use a service bus pub/sub for this? I don't know about your app's architecture and how content is uploaded; this may or may not work for you.

Answer 2

I have code that does this. See https://github.com/smarx/noderole/blob/master/WebRole/Sync/OneWayBlobSync.cs.