SignalR - Works when deployed to Server, but stops after a few hours (MVC)

Question

EDIT: Look at the bottom of this post for updates.

My SignalR implementation works perfectly on my local system. But when I deployed it out to my server it doesnt seem to work. Its an MVC project.

My signalR jQuery is as follows:

var clientHub = $.connection.gamehub;

$(function () {

    var signalRHubInitialized = false;

    var image = $("#Ico");

    var count = 0;

    initializeSignalRHubStore();

    function initializeSignalRHubStore() {

        if (signalRHubInitialized)
            return;

        try {

            clientHub.client.broadcastMessage = function (message) {
                if (message === "Refresh")
                    reloadIndexPartial();
            };

            $.connection.hub.start().done(function () {
                clientHub.server.initialize($("#NotifierEntity").val());
                signalRHubInitialized = true;
            });

        } catch (err) {
            signalRHubInitialized = false;
        }
    };

    function reloadIndexPartial() {
        //$.post('@(Url.Action("LivePartial", "Scrim", null, Request.Url.Scheme))')
        var id = $("#SeriesDetail_Id").val();
        $.post('/Scrim/LivePartial/' + id)
            .done(function (response) {
                try {
                    count = count + 1;
                    var favicon = new Favico({
                        animation: 'pop',
                        image: image
                    });
                    favicon.badge(count);
                }
                catch (exception) {

                }
                $("#summary-wrapper").html("");
                $("#summary-wrapper").html(response);
                if (!signalRHubInitialized)
                    initializeSignalRHubStore();
            });
    };
});

I downloaded Fiddler to see what was going on:

/signalr/hubs returned a HTTP200

GET http://sitename.com/signalr/hubs HTTP/1.1
Host: sitename.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Accept: */*
Referer: http://sitename.com/scrim/Live/2835
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: _gat=1; _ga=GA1.2.1342148401.1475084375; _gid=GA1.2.2092796788.1503865866

negotiate returned at HTTP200

GET http://sitename.com/signalr/negotiate?clientProtocol=1.5&connectionData=%5B%7B%22name%22%3A%22gamehub%22%7D%5D&_=1505151041506 HTTP/1.1
Host: sitename.com
Connection: keep-alive
Accept: text/plain, */*; q=0.01
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Content-Type: application/json; charset=UTF-8
Referer: http://sitename.com/scrim/Live/2835
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: _gat=1; _ga=GA1.2.1342148401.1475084375; _gid=GA1.2.2092796788.1503865866

connect, didnt return anything

GET http://sitename.com/signalr/connect?transport=serverSentEvents&clientProtocol=1.5&connectionToken=S8rqz2NPvVSJxbS1%2FpLm7yHTinGHWK1SnAwh8IfYA%2BP7nVb9RV%2FJzSFsf8Q%2BTv6Z%2Fae%2FIoZKlHKyeTxaEn3obg%2FVViYTB5HZxnrvKvtBZtQopvGPdj1i4o8Z9wGlCz3%2F&connectionData=%5B%7B%22name%22%3A%22gamehub%22%7D%5D&tid=10 HTTP/1.1
Host: sitename.com
Connection: keep-alive
Accept: text/event-stream
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Referer: http://sitename.com/scrim/Live/2835
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: _gat=1; _ga=GA1.2.1342148401.1475084375; _gid=GA1.2.2092796788.1503865866

start returned a HTTP200

GET http://sitename.com/signalr/start?transport=serverSentEvents&clientProtocol=1.5&connectionToken=S8rqz2NPvVSJxbS1%2FpLm7yHTinGHWK1SnAwh8IfYA%2BP7nVb9RV%2FJzSFsf8Q%2BTv6Z%2Fae%2FIoZKlHKyeTxaEn3obg%2FVViYTB5HZxnrvKvtBZtQopvGPdj1i4o8Z9wGlCz3%2F&connectionData=%5B%7B%22name%22%3A%22gamehub%22%7D%5D&_=1505151041507 HTTP/1.1
Host: sitename.com
Connection: keep-alive
Accept: text/plain, */*; q=0.01
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Content-Type: application/json; charset=UTF-8
Referer: http://sitename.com/scrim/Live/2835
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: _gat=1; _ga=GA1.2.1342148401.1475084375; _gid=GA1.2.2092796788.1503865866

Send returned a HTTP200

POST http://sitename.com/signalr/send?transport=serverSentEvents&clientProtocol=1.5&connectionToken=S8rqz2NPvVSJxbS1%2FpLm7yHTinGHWK1SnAwh8IfYA%2BP7nVb9RV%2FJzSFsf8Q%2BTv6Z%2Fae%2FIoZKlHKyeTxaEn3obg%2FVViYTB5HZxnrvKvtBZtQopvGPdj1i4o8Z9wGlCz3%2F&connectionData=%5B%7B%22name%22%3A%22gamehub%22%7D%5D HTTP/1.1
Host: sitename.com
Connection: keep-alive
Content-Length: 2227
Accept: text/plain, */*; q=0.01
Origin: http://sitename.com
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://sitename.com/scrim/Live/2835
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.8
Cookie: _gat=1; _ga=GA1.2.1342148401.1475084375; _gid=GA1.2.2092796788.1503865866

data=%7B%22H%22%3A%22gamehub%22%2C%22M%22%3A%22Initialize%22%2C%22A%22%3A%5B%22%7B%5C%22SqlQuery%5C%22%3A%5C%22SELECT+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BId%5D+AS+%5BId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BGameGuid%5D+AS+%5BGameGuid%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BDate%5D+AS+%5BDate%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BTeamOneScore%5D+AS+%5BTeamOneScore%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BTeamZeroScore%5D+AS+%5BTeamZeroScore%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BTeamOneId%5D+AS+%5BTeamOneId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BTeamZeroId%5D+AS+%5BTeamZeroId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BGameVariantId%5D+AS+%5BGameVariantId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BMapId%5D+AS+%5BMapId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BDuration%5D+AS+%5BDuration%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BSeriesId%5D+AS+%5BSeriesId%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BResult%5D+AS+%5BResult%5D%2C+%5C%5Cr%5C%5Cn++++%5BExtent1%5D.%5BActive%5D+AS+%5BActive%5D%5C%5Cr%5C%5Cn++++FROM+%5Bdbo%5D.%5BGame%5D+AS+%5BExtent1%5D%5C%5Cr%5C%5Cn++++WHERE+(%5BExtent1%5D.%5BActive%5D+%3D+1)+AND+(%5BExtent1%5D.%5BSeriesId%5D+%3D+%40p__linq__0)%5C%22%2C%5C%22SqlConnectionString%5C%22%3A%5C%22Data+Source%3DWIN-1J1JAEOEU33%3BInitial+Catalog%3DSiteName%3BIntegrated+Security%3DTrue%3BMultipleActiveResultSets%3DTrue%3B%5C%22%2C%5C%22SqlParameters%5C%22%3A%5B%7B%5C%22CompareInfo%5C%22%3A0%2C%5C%22XmlSchemaCollectionDatabase%5C%22%3A%5C%22%5C%22%2C%5C%22XmlSchemaCollectionOwningSchema%5C%22%3A%5C%22%5C%22%2C%5C%22XmlSchemaCollectionName%5C%22%3A%5C%22%5C%22%2C%5C%22DbType%5C%22%3A11%2C%5C%22LocaleId%5C%22%3A0%2C%5C%22ParameterName%5C%22%3A%5C%22p__linq__0%5C%22%2C%5C%22Precision%5C%22%3A0%2C%5C%22Scale%5C%22%3A0%2C%5C%22SqlDbType%5C%22%3A8%2C%5C%22SqlValue%5C%22%3A%7B%5C%22IsNull%5C%22%3Afalse%2C%5C%22Value%5C%22%3A2835%7D%2C%5C%22UdtTypeName%5C%22%3A%5C%22%5C%22%2C%5C%22TypeName%5C%22%3A%5C%22%5C%22%2C%5C%22Value%5C%22%3A2835%2C%5C%22Direction%5C%22%3A1%2C%5C%22IsNullable%5C%22%3Afalse%2C%5C%22Offset%5C%22%3A0%2C%5C%22Size%5C%22%3A0%2C%5C%22SourceColumn%5C%22%3A%5C%22%5C%22%2C%5C%22SourceColumnNullMapping%5C%22%3Afalse%2C%5C%22SourceVersion%5C%22%3A512%7D%5D%7D%22%5D%2C%22I%22%3A0%7D

I have added this to my web config:

<modules runAllManagedModulesForAllRequests="true"></modules>

Looking through all the responses it seems that everything is working correctly but the page Im on is not being updated when a new entry has been added to the database.

On my local development system my project is set up using IIS it works flawlessly.

Could anyone point me in the right direction please.

EDIT: I have got it working on the server now. But it seems that it works right after it has been deployed for a few hours. Then after that it seems to stop working. So I have to assume that the signalr connection is being disposed at some stage and now getting reinstated?

Here is my RegisterServices class:

private static IContainer RegisterServices(ContainerBuilder builder)
{
    builder.RegisterControllers(Assembly.GetExecutingAssembly());

    builder.RegisterApiControllers(Assembly.GetExecutingAssembly());

    builder.RegisterType<ContextEntities>()
           .As<DbContext>()
           .InstancePerRequest();

    builder.RegisterType<DbFactory>()
        .As<IDbFactory>()
        .InstancePerRequest();

    builder.RegisterType<UnitOfWork>()
        .As<IUnitOfWork>()
        .InstancePerRequest();

    // Services

    builder.RegisterType<MembershipService>()
    .As<IMembershipService>()
    .InstancePerRequest();

    builder.RegisterType<CacheService>()
    .As<ICacheService>()
    .InstancePerRequest();

    builder.RegisterType<GameHub>().ExternallyOwned();

    Container = builder.Build();

    return Container;
}

Here is a page where signalr is used: http://halodatahive.com/Scrim/Live/2845

I seem to be losing reference to the signalr connection after a few hours after a deployment.

EDIT: If I recycle my APP POOL the page with signalR starts working again.

Answer 1

This is what I ended up using to resolve the issue. It seems that after around 1 hour it was getting disconnected some how. I put this code in a few hours ago and it still seems to be working. Thanks to @Noren for all their help in chat earlier!

EDIT: This did not seem to solve the problem unfortunately.

$.connection.hub.disconnected(function() {
    setTimeout(function() {
      $.connection.hub.start();
    }, 5000); // Restart connection after 5 seconds.
});

EDIT: Thought I would give an update as to how I got this working. Instead of using SqlDependency to trigger the SignalR I just called Clients.All.broadcastMessage("Refresh"); on the scheduled task I have running on the server when _unitOfWork.Commit() was called.

Something was causing SqlDependency to stop working and the only way to get it to pick it up again was to recycle the app pool.

Answer 2

I've seen something like this before. In my case it was RabbitMQ events that were lost because IIS was spinning down the application.

Is your application is not being hit very frequently? IIS has a tendency to spin down applications that it doesn't think it needs in order to save resources. That might be why it only stops working after a few hours and you can recycle to bring it back up.

See this answer.