Is it possible to use MassTransit 3 with Azure Service Bus without Manage permission policy?

Question

I spent some days testing MassTransit 3.1.2 to see if we can use it with Azure Service Bus in our applications.

I made a sample with two console applications using MassTransit.AzureServiceBus (3.1.2) : one publisher and one suscriber.

It works well. When I start the applications, the entities (queues, topic, subscriptions) are created automatically on my namespace on Azure.
That's nice when you are testing thing but in production, I don't want the application to be allowed to create entities. We want to create them upfront.

To try that, I thought It was a good idea to connect to the bus using SAS policy with "Send" or "Listen" permissions only (before I was using a namespace policy with "Manage" permission).

Now I'm struggling on this point, I can't get it to work, I'm always getting 401 errors Manage claim is required for this operation if I don't use a policy with "Manage" permissions.
I tried setting the policy on the namespace or the entities directly without success.

After that I analyzed the stack trace exception (useless part omitted with [...]) :

System.UnauthorizedAccessException: Le serveur distant a retourné une erreur : (401) Non autorisé. Manage claim is required for this operation. TrackingId:2ca420e3-aac6-467c-bacb-6e051dbc3e39_G47,TimeStamp:1/29/2016 11:20:41 PM ---> System.Net.WebException: Le serveur distant a retourné une erreur : (401) Non autorisé.
   à System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   à Microsoft.ServiceBus.Messaging.ServiceBusResourceOperations.GetAsyncResult`1.<GetAsyncSteps>b__3c(GetAsyncResult`1 thisPtr, IAsyncResult r)
   à Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
   --- Fin de la trace de la pile d'exception interne ---

Server stack trace:

Exception rethrown at [0]:
   à Microsoft.ServiceBus.Common.ExceptionDispatcher.Throw(Exception exception)
   à Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
   à Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
   à Microsoft.ServiceBus.Messaging.ServiceBusResourceOperations.EndGet[TEntityDescription](IAsyncResult asyncResult, String[]& resourceNames)
   à Microsoft.ServiceBus.NamespaceManager.EndGetQueue(IAsyncResult result)
   à System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
[...]
   à MassTransit.AzureServiceBusTransport.NamespaceManagerExtensions.<CreateQueueSafeAsync>d__1.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
[...]
   à MassTransit.AzureServiceBusTransport.Pipeline.PrepareReceiveQueueFilter.<Send>d__5.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
[...]
   à MassTransit.AzureServiceBusTransport.ServiceBusReceiveTransport.<>c__DisplayClass12_0.<<Receiver>b__0>d.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
[...]
   à MassTransit.Internals.Extensions.TaskExtensions.<WithCancellation>d__0`1.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception 
[...]
   à MassTransit.MassTransitBus.<StartAsync>d__30.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   à MassTransit.MassTransitBus.<StartAsync>d__30.MoveNext()
--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---
   à System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   à System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   à System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   à MassTransit.Util.TaskUtil.Await[T](Func`1 taskFactory, CancellationToken cancellationToken)
   à MassTransit.MassTransitBus.MassTransit.IBusControl.Start()

I found out that the line with MassTransit.AzureServiceBusTransport.NamespaceManagerExtensions.CreateQueueSafeAsync to be really interesting because I was able to look at the MassTransit source code to see what it was doing. I saw that it was doing some calls using the NamespaceManager to get the queue or topic.

Since this class is named NamespaceManager, I thought that would mean you need "Manage" permission anyway.
To try that, I made a basic console application using only the Azure SDK to make some calls to the NamespaceManager using a policy with only Listen or Send permissions : I got 401 errors on all the calls I tried. Adding Manage permission worked.

I didn't find anything about this assumption in the Azure documentation or maybe I missed something.

Final question :

Is there a way to use MassTransit on Azure Service Bus with a Send or Listen policy only ? Did I miss something and I'm heading the wrong way ?

Answer 1

Because MassTransit is responsible for managing the topology of the service bus namespace, including creating topics and queues as well as creating and binding subscriptions, the manage permission is required.

While you might think it's a great idea to create everything in production manually and leave that permission off your application, you will always spend time figuring out why things are broken in production and frustrate your engineers. I speak from experience on this one -- it's why we require the permission.

There are also auto-delete queues created for bus management, which again requires the manage permission.

UPDATE: Manage is still required, however, you might be able to get around it. If you can create topics, queues, and subscriptions in advance, and you configure MassTransit to not create topics, queues, or subscriptions, and probably to not publish faults (unless you're going to create those topics as well), and not use error or skipped queues.

For example, this configuration would basically limit the use of MassTransit to a queue only:

var bus = Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
    cfg.Host(...);

    cfg.ReceiveEndpoint("existing-queue", e =>
    {
        e.PublishFaults = false;
        e.ConfigureConsumeTopology = false;

        e.ConfigureDeadLetterQueueErrorTransport();
        e.ConfigureDeadLetterQueueDeadLetterTransport();

        e.Consumer(...);
    });
});

Messages sent to the queue directly (destinationAddress of queue:existing-queue) would be consumed by the receive endpoint, and error/skipped messages would be moved to the Azure dead-letter queue. You could only call publish if the topic name matching the message type exists (or you could use topic:existing-topic-name as the destination address).