Multiple input queues in one rebus process

Question

I'm writing a "multi-workers" application using top-shelf and rebus.

My idea is to use the MyWorker1Namespace - MyWorker1Namespace.Messages, MyWorker2Namespace - MyWorker2Namespace.Messages pattern.

I would like to run the application without spanning multiple process, instead I would like to configure the application with moultiple input queues in order to be ready to split it up to multiple processes if necessary.

Is there any way to declare multiple input queues and multiple worker threads in one application using Rebus?

I guess configuration should be something like this:

<rebus inputQueue="MyWorker1Namespace.input" errorQueue="MyWorker1Namespace.error" workers="1" maxRetries="5"> </rebus>
<rebus inputQueue="MyWorker2Namespace.input" errorQueue="MyWorker2Namespace.error" workers="1" maxRetries="5"> </rebus>
...

Answer 1

Since the Rebus app.config XML is kind of optimized for one-bus-instance-per-process scenarios, you cannot configure multiple buses fully in XML, but there's nothing that keeps you from starting multiple bus instances inside the same process.

I've often done that e.g. in Azure worker roles where I want to host multiple logical endpoints without incurring the cost of physically separate deployments, and I've also sometimes done it with Topshelf-hosted Windows Services.

Usually, my app.config ends up with the following XML:

<rebus workers="1">
    <add messages="SomeAssembly.Messages" endpoint="someEndpoint.input"/>
    <add messages="AnotherAssembly.Messages" endpoint="anotherEndpoint.input"/>
</rebus>

thus allowing me to configure the default number of workers per bus and the endpoint mappings once and for all. Then, when my application starts up, it will keep an IoC container per bus for the duration of the application lifetime - with Windsor, I usually end up with a general bus installer that has parameters for the queue names, which allows me to configure Windsor like this:

var containers = new List<IWindsorContainer> {
    new WindsorContainer()
        // always handlers first
        .Install(FromAssembly.Containing<SomeEndpoint.SomeHandler>())

        // and then the bus, which gets started at this point
        .Install(new RebusInstaller("someEndpoint.input", "error"))

        // and then e.g. background timers and other "living things"
        .Install(new PeriodicTimersInstannce()),

    new WindsorContainer()
        .Install(FromAssembly.Containing<AnotherEndpoint.AnotherHandler>())
        .Install(new RebusInstaller("anotherEndpoint.input", "error"))
};

// and then remember to dispose each container when shutting down the process

where the RebusInstaller (which is a Windsor mechanism) basically just puts a bus with the right queue names into the container, e.g. like this:

Configure.With(new WindsorContainerAdapter(container))
    .Transport(t => t.UseMsmq(_inputQueueName, _errorQueueName))
    .(...) etc
    .CreateBus().Start();

I like the idea that each IoC container instance functions as a logically independent application on its own. This way it would be easy to break things apart some time in the future if you want to e.g. be able to deploy your endpoints independently.

I hope this provides a bit of inspiration for you :) please don't hesitate to ask if you need more pointers.