Am I expecting too much scalability from azure functions?

Question

I have an azure function running on the consumption plan that gets triggered by a service bus topic then simply adds another message to a queue (so basically does no processing, just IO). The trigger topic is being filled at a rate of ~1000/second. I naively thought that the function could easily keep pace, but it is in fact hopelessly overwhelmed and the topic subscription fills very quickly. I have it running for hours so I suspect it is fully scaled out.

How much performance should I expect from functions? Is throughput on the order of thousands per second out of the question?

edit: I am regularly seeing this error in the logs:

The connection attempt lasted for a time span of 00:00:00. TCP error code 10013: An attempt was made to access a socket in a way forbidden by its access permissions.

Looks like functions don't play well with service bus at scale?

edit 2 for solution Replacing this binding:

public static async Task Run(BrokeredMessage msgin, Binder binder, TraceWriter log)
{
 var collector = await binder.BindAsync<IAsyncCollector<BrokeredMessage>>(
                new ServiceBusAttribute("my-queue"));

...
}

with this:

public static IAsyncCollector<BrokeredMessage> collector;
public static async Task Run(BrokeredMessage msgin, Binder binder, TraceWriter log)
{
     collector = collector ?? await binder.BindAsync<IAsyncCollector<BrokeredMessage>>(
                new ServiceBusAttribute("my-queue"));


    ...
}

prevented socket exhaustion and the function was able to keep pace with the producer no problem.

Answer 1

I had the same issue in my Azure Function. The root cause was that I was exhausting all ports because I was creating a TCP connection inside Run method.

So when there were many parallel executions on the same machine, they all created connection, eventually exhausting all ports.

In that situation, connection was created as part of creation of ServiceBus client, and moving it to static variable resolved the issue.