Correct way to host a *stable* WCF MSMQ windows service

Question

I have an application that uses MSMQ for asynchronous processing of certain things.

I use WCF to put messages on to the queue and have a WCF MSMQ listener (a windows service) to receive messages and deal with them.

My problem is keeping this stable. What is the correct way to deal with (for example) the queue server (which is a separate box) going down? The other day this happened and the service just sat there - no exceptions were thrown, it just stopped receiving messages. I would want it to throw an exception when the queue server went down and then re-try to connect to it until it is able.

I have also noticed that executing a "stop" on the service often causes it to hang for quite a while before it finally stops.

Any code suggests or criticisms would be welcome. Obviously I did Google for this first, but most examples show me pretty much what I already have and I would like to make my system more robust than that.

Currently I have this:

(Note: IMyExampleServiceContract is my WCF service contract and QueueHandler is what implements it)

namespace xyz.MyExample.MSMQListener
{
    /// <summary>
    /// The class that handles starting and stopping of the WCF MSMQ Listener windows service.
    /// It will respond to start and stop commands from within the windows services administration snap-in
    /// It creates a WCF NetMsmqBinding that watches a particular queue for messaages defined by a contract
    /// in the ServiceContracts project.
    /// </summary>
    public partial class MsmqListenerService : ServiceBase
    {
        /// <summary>
        /// The WCF service host
        /// </summary>
        private ServiceHost _serviceHost;

        /// <summary>
        /// Defines the maximum size for a WCF message
        /// </summary>
        private const long MaxMessageSize = 1024 * 1024 * 1024; // 1 gb
        /// <summary>
        /// Defines the maximum size for a WCF array
        /// </summary>
        private const int MaxArraySize = 1024 * 1024 * 1024; // 1 gb

        /// <summary>
        /// The queue name
        /// </summary>
        private readonly string _queueName;
        /// <summary>
        /// The queue server
        /// </summary>
        private readonly string _queueServer;

        /// <summary>
        /// Initializes a new instance of the <see cref="MsmqListenerService"/> class.
        /// </summary>
        public MsmqListenerService()
        {
            InitializeComponent();
            using (ConfigManager config = new ConfigManager())
            {
                _queueName = config.GetAppSetting("QueueName");
                _queueServer = config.GetAppSetting("QueueServer");
            }
        }

        /// <summary>
        /// When implemented in a derived class, executes when a Start command is sent to the service by the Service Control Manager (SCM) or when the operating system starts (for a service that starts automatically). Specifies actions to take when the service starts.
        /// <para>
        /// The logic in this method creates a WCF service host (i.e. something that listens for messages) using the <see cref="IMyExampleServiceContract"/> contract.
        /// The WCF end point is a NetMSMQBinding to the MyExample MSMQ server/queue.
        /// It sets up this end point and provides a class to handle the messages received on it.
        /// The NetMSMQBinding is a Microsoft WCF binding that handles serialisation of data to MSMQ. It is a ms proprietary format and means that the message on the queue
        /// can only be read by a WCF service with the correct contract information.
        /// </para>
        /// </summary>
        /// <param name="args">Data passed by the start command.</param>
        protected override void OnStart(string[] args)
        {
            try
            {
                Logger.Write("MyExample MSMQ listener service started.", StandardCategories.Information);

                Uri serviceUri = new Uri("net.msmq://" + QueueServer + QueueName);

                NetMsmqBinding serviceBinding = new NetMsmqBinding();
                serviceBinding.Security.Transport.MsmqAuthenticationMode = MsmqAuthenticationMode.None;
                serviceBinding.Security.Transport.MsmqProtectionLevel = System.Net.Security.ProtectionLevel.None;
                serviceBinding.MaxReceivedMessageSize = MaxMessageSize;
                serviceBinding.ReaderQuotas.MaxArrayLength = MaxArraySize;

                //QueueHandler implements IMyExampleServiceContract
                _serviceHost = new ServiceHost(typeof(QueueHandler));
                _serviceHost.AddServiceEndpoint(typeof(IMyExampleServiceContract), serviceBinding, serviceUri);

                _serviceHost.Open();
                Logger.Write("MyExample MSMQ listener service completed OnStart method.", StandardCategories.Information);
            }
            catch (Exception ex)
            {
                ExceptionReporting.ReportException(ex, "DefaultExceptionPolicy");
                throw;
            }
        }

        /// <summary>
        /// Gets the name of the queue to send to. 
        /// This is retrieved from the application settings under QueueName
        /// </summary>
        private string QueueName
        {
            get { return _queueName; }
        }

        /// <summary>
        /// Gets the name of the queue server to send to. 
        /// This is retrieved from the application settings under QueueServer
        /// </summary>
        private string QueueServer
        {
            get { return _queueServer; }
        }

        /// <summary>
        /// When implemented in a derived class, executes when a Stop command is sent to the service by the Service Control Manager (SCM). Specifies actions to take when a service stops running.
        /// </summary>
        protected override void OnStop()
        {
            if (_serviceHost != null)
            {
                _serviceHost.Close();
                _serviceHost = null;
            }
        }

        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        public static void Main()
        {
            //Code will have to be compiled in release mode to be installed as a windows service
            #if (!DEBUG)
                try
                {
                    Logger.Write("Attempting to start queue listener service.", StandardCategories.Information);
                    ServiceBase[] ServicesToRun;
                    ServicesToRun = new ServiceBase[]
                            {
                            new MsmqListenerService()
                            };
                    ServiceBase.Run(ServicesToRun);
                    Logger.Write("Finished ServiceBase.Run of queue listener service.", StandardCategories.Information);
                }
                catch (Exception e)
                {
                    ExceptionReporting.ReportException(e, "DefaultExceptionPolicy");
                    throw;
                }
            #else
                //This allows us to run from within visual studio
                MsmqListenerService service = new MsmqListenerService();
                service.OnStart(null);
                System.Threading.Thread.Sleep(System.Threading.Timeout.Infinite);
            #endif

        }
    }
}

Answer 1

I'm not sure why your service host is hanging, but I can definitely think of a couple of things to try to make it more reliable:

• I'd make sure to hook into the Faulted event of the service host. That's usually a good place to recognize that you need to respawn your host again.
• I'd set up a way for the service to ping itself, by having a special health-status queue on the remote queue server and have a second custom WCF service listening on that queue. Then I'd have the service host just fire messages into that queue regularly and check that:

a) it can send them successfully and

b) that the messages are being picked up and processed by the local WCF health service listening on that queue. That could be used to detect some possible error conditions.