How to work with streaming data using azure functions?

Question

I have created a realtime solution using Azure services. It works based on the following architecture. NSE(National Stock Exchanges) -> AzureFunction -> EventHub ->Azure Stream Analytics -> Power BI. In the Azure Function, I wrote the following code.

module.exports = async function (context, myTimer) {
    var API = require('indian-stock-exchange')
    var NSEAPI = API.NSE;
    let promise = new Promise((resolve,reject)=>{
        NSEAPI.getGainers()
        .then(function (response) { 
            context.log(response['data']); //return the api data
            resolve(response['data'])
    });
    });
    let result = await promise;
    result = JSON.stringify(result['data'])
    return result 
};

I'm using the library here and that calls through the HTTP request but for the streaming, this not a right way, can you guys share any suggestion on my code or any changes on architecture workflow.

Thanks in advance.

Answer 1

As mentioned in some of the comments, the challenge here is that I think you want a persistent “web socket” to the stock exchange constantly pushing data to an Event Hub. Functions by their nature or more short-lived. So here the code works, but does a bit of a “wake up, do work, go to sleep, repeat” pattern.

For a persistent connection you may have better luck looking into using something like Azure WebJobs, or just running a web app inside of Azure App Services that will be running 24/7 and could create a persistent feed from the stock data to Event Hubs.