How to make a Hangfire instance run its own jobs only?

Question

I have a couple instances of Hangfire running on two servers using the same database. Each instance submits jobs to be run based on some criteria based on server name so that no two instances run the same job. I noticed that they are running the same jobs which means when an instance is running it's picking any job in the queue in the database regardless if it submitted the job or not. I thought in the latest version 1.6.x, each job is unique. It seems this doesn't mean it runs only on the instance that created it?

How do I get each instance to run jobs it submitted only?

Answer 1

You need to use queues to select which server handles specific jobs.

The idea is to categorize jobs by specifying a queue. Then for each server you will specify which queue(s) they watch.

The only problem with this, in my opinion, is that choosing the queue for a job is not straightforward (unless you are working with RecurringJobs).

Server Configuration

When you start the Hangfire instance for a server, use the Queues BackgroundJobServerOptions as per the documentation:

app.UseHangfireServer(new BackgroundJobServerOptions()
    {
        // order defines priority
        // beware that queue names should be lowercase only
        Queues = new [] { "critical", "default", "myqueueformyserver" } 
    });

Selecting a queue for a job

There are two cases:

1. RecurringJobs: RecurringJob.AddOrUpdate("MyFirstRecurringJob", () => myClass.myMethod(), Cron.Minutely(), null, "myqueueformyserver");

2. BackgroundJobs: you cannot specify the queue for the job at enqueue time (Hangfire.BackgroundJob.Enqueue(() => myClass.myMethod());) there is no option for this. The solution is to use a method or class attribute. Hangfire provides a QueueAttribute:
   [Queue("myqueueformyserver")] public void myMethod() { }

If I understand your requirements, the static QueueAttribute will not fit you as you want to dynamically assign the queue. I had the same situation and created my own attribute inspired by the code of the QueueAttribute.

It looks something like that (adapt to your willing/needs)

public class MyQueueAttribute : JobFilterAttribute, IElectStateFilter
{
    public MyQueueAttribute(string paramQueue)
    {
        ParamQueue = paramQueue;
    }

    public string ParamQueue { get; }

    public void OnStateElection(ElectStateContext context)
    {
        var enqueuedState = context.CandidateState as EnqueuedState;
        if (enqueuedState != null)
        {
            enqueuedState.Queue = string.Concat(Environment.MachineName.ToLower(), 
                                                ParamQueue);
        }
    }
}