How to index Azure storage data to elastic cloud

Question

I am new to elastic search. I have data stored in Azure storage and I want to index it using elasticsearch. I have created a cluster at https://cloud.elastic.co. Do I need to create a service which will index the data in elastic cloud and then users can use/search this data using elastic search? How to index the data to elastic cloud using asp.net MVC?

Please suggest.

Answer 1

One way to approach this would be to write a console application that

1. pulls data from Azure storage using the Storage client in the WindowsAzure.Storage nuget package or similar
2. transforms data into documents according to your domain needs
3. bulk indexes documents into Elasticsearch in Elastic Cloud using the .NET Elasticsearch client NEST

If data will be updated in Azure storage and will need to be frequently indexed into Elasticsearch, consider making the console application an Azure Web Job.

Another approach would be to use Logstash in conjunction with the input plugin for Azure Storage blobs.