Hangfire keeps running SQL queries even when inactive

Question

I'm developing an ASP.net MVC 5 web site and I'm using Hangfire for scheduling some tasks, in this case just one every 3 min. I know for a fact that it takes only a few seconds to run such task (and the DB query associated with it).

The problem I'm facing is that it seems as if Hangfire has my SQL Server running "something" (I don't know what) and I can see in the SQL Server Activity Monitor that my CPU stays always at 20+% usage and there are Database I/O operations at (1.2 MB/sec average). I know it is Hangfire because when I don't initialize it the Activity Monitor (and Task Manager) shows no extra overhead. I have even gone as far as to remove all scheduled tasks and anything/everything that Hangfire can run and still the problem persists.

I can't go to production like this for I fear it may cause performance issues. Any help will be most appreciated, thanks in advance

Answer 1

I investigated this a bit on my own server with MVC app + hangfire. Indeed my CPU usage is at 20-25% too. So I searched for a suitable monitor app, installed a nifty little tool called "SQLRanger" and found that the top query by far is this:

update top (1) HangFire.JobQueue set FetchedAt = GETUTCDATE()
output INSERTED.Id, INSERTED.JobId, INSERTED.Queue
where FetchedAt is null
and Queue in (@queues1)

So it is basically hangfire checking for jobs waiting to be performed. So far I haven't encountered any performance issues or lags though.

The issue is obviously caused - and remedied - by adjusting the polling interval, see the respective section of http://docs.hangfire.io/en/latest/configuration/using-sql-server.html

The default interval is 15 seconds, which ensures prompt processing of jobs but also constant server load. In non-time-critical applications a higher interval (1 min, 5 mins etc) should be OK. Know what you need and react to it: need near immediate job-processing or low server load? If the former, keep the interval short and think about upsizing the server if needed; if the latter, increase the interval to the highest acceptable minimum.

I need the former and will keep an eye on the server whether it can bear the load.

Answer 2

I have also noticed lots of queries being issued when some dashboard view was opened and it seems that both dashboard stats polling interval and sql server polling interval must be set to a reasonable enough value to avoid flooding the SQL Server (the following is from an ASP.NET Core 2.0 implementation with Hangfire 1.7):

services.AddHangfire(opt => opt.UseSqlServerStorage(Configuration.GetConnectionString("Default"),
    new SqlServerStorageOptions
    {
        CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
        QueuePollInterval = TimeSpan.FromSeconds(30),
        UseRecommendedIsolationLevel = true,
        UsePageLocksOnDequeue = true,
        DisableGlobalLocks = true
    }));

app.UseHangfireDashboard("/hangfire", new DashboardOptions
{
    Authorization = new [] {new HangfireDashboardAuthorizationFilter()},
    StatsPollingInterval = 30000
});