Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure Functions: configure blob trigger only for new events

I have about 800k blobs in my azure storage. When I create azure function with a blobTrigger it starts to process all blobs that I have in the storage. How can I configure my function to be triggered only for new and updated blobs?

like image 771
ebashmakov Avatar asked Dec 07 '16 02:12

ebashmakov


People also ask

How do you use a Blob trigger in Azure function?

For more information, see Tutorial: Trigger Azure Functions on blob containers using an event subscription. Blob name string is manually added to a storage queue when a blob is added to the container. This value is passed directly by a Queue Storage trigger to a Blob Storage input binding on the same function.

Can Azure functions have multiple triggers?

There are no plans to support multiple triggers per Function. You will have to create a Function for each EventHub. If there is common code that may be shared between Functions, you may move them to a helper method that can be called from each Function.


2 Answers

There is no way to do this currently. Internally we track which blobs we have processed by storing receipts in our control container azure-webjobs-hosts. Any blob not having a receipt, or an old receipt (based on blob ETag) will be processed (or reprocessed). That's why your existing blobs are being processed - they don't have receipts. BlobTrigger is currently designed to ensure that ALL blobs in a container matching the path pattern are eventually processed, and reprocessed any time they are updated.

If you feel passionately about this, you can log a feature request in our repo here with details on your scenario.

like image 98
mathewc Avatar answered Sep 27 '22 16:09

mathewc


The way I get around this is to set Metadata on a processed Blob (e.g Status = Complete). When the trigger gets fired I first check for this piece of Metadata and return the function if it is already set.

The downside to this is that the update of the Metadata will trigger an additional execution of the function

like image 22
Steve Kay Avatar answered Sep 27 '22 16:09

Steve Kay