Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Quartz.net Scheduler.Shutdown(true) not killing jobs

Tags:

c#

quartz.net

I have a running quartz job and terminating my BackgroundService, for some reason despite calling scheduler.Shutdown(true) the job remains running.

Even when looping through and interrupting jobs, the program closes if before the threads exit.

Aside from my code below, would i be looking at writing a custom IScheduler to ensure runnings jobs are halted on shutdown?

This is my IJob Execute method:

public async Task Execute(IJobExecutionContext context)
{
    var cancellationToken = context.CancellationToken;

    while (cancellationToken.IsCancellationRequested == false)
    {
        // Extension method so we catch TaskCancelled exceptions.
        await TaskDelay.Wait(1000, cancellationToken);
        Console.WriteLine("keep rollin, rollin, rollin...");
    }
    Console.WriteLine("Cleaning up.");
    await Task.Delay(1000);
    Console.WriteLine("Really going now.");
}

This is my shutdown loop (calling shutdown directly doesn't interrupt any running jobs):

internal class QuartzHostedService : IHostedService
{
    // These are set by snipped constructor.
    private readonly IJobSettings jobSettings;
    private readonly ILogger logger;
    private readonly IScheduler scheduler;

    private async Task AddJobsToScheduler(CancellationToken cancellationToken = default)
    {
        var schedule = SchedulerBuilder.Create();

        var downloadJob = JobBuilder.Create<StreamTickersJob>().Build();

        var downloadJobTrigger = TriggerBuilder
            .Create()
            .ForJob(downloadJob)
            .WithDailyTimeIntervalSchedule(
                x => x.InTimeZone(serviceTimeZone)
                    .OnEveryDay()
                    .StartingDailyAt(new TimeOfDay(8,0))
                    .EndingDailyAt(new TimeOfDay(9,0)))
            .Build();

        await this.scheduler.ScheduleJob(downloadJob, downloadJobTrigger, cancellationToken);
    }

    public QuartzHostedService(IJobSettings jobSettings, IScheduler scheduler, ILogger<QuartzHostedService> logger)
    {
        this.jobSettings = jobSettings;
        this.scheduler = scheduler;
        this.logger = logger;
    }

    public async Task StartAsync(CancellationToken cancellationToken)
    {
        this.logger.LogInformation("Quartz started...");
        await AddJobsToScheduler(cancellationToken);
        await this.scheduler.Start(cancellationToken);
    }

    public async Task StopAsync(CancellationToken cancellationToken)
    {
        await this.scheduler.PauseAll(cancellationToken);

        foreach (var job in await this.scheduler.GetCurrentlyExecutingJobs(cancellationToken))
        {
            this.logger.LogInformation($"Interrupting job {job.JobDetail}");
            await this.scheduler.Interrupt(job.JobDetail.Key, cancellationToken);

        }
        await this.scheduler.Shutdown(cancellationToken);
    }
}

I can confirm IHost is not killing my app abruptly (at least not for a couple of seconds test pause) as I set a breakpoint at the end of the main program as below:

public static void Main(string[] args)
{
    // Wrap IHost in using statement to ensure disposal within scope.
    using (var host = CreateHostBuilder(args)
                                .UseSerilog<Settings>(Settings.Name)
                                .UseConsoleLifetime()
                                .Build()
                                .UseSimpleInjector(container))
    {
        // Makes no difference if I shutdown jobs here.
        // var lifetime = container.GetInstance<IHostApplicationLifetime>();            
        // lifetime.ApplicationStarted.Register(async () => { });
        // lifetime.ApplicationStopping.Register(async () => { });

        var logger = container.GetInstance<ILogger<Program>>();

        try
        {
            host.Run();
        }
        catch (Exception ex)
        {
            logger.LogCritical(ex, ex.Message);
        }

        // We reach here, whilst Jobs are still running :(
        logger.LogDebug($"Finish {nameof(Main)}().");
    }
}

I have also added from what I have found online the below, but still it doenst wait on shutdown:

var props = new NameValueCollection
{
    {"quartz.scheduler.interruptJobsOnShutdownWithWait", "true"},
};

var scheduler = AsyncContext.Run(async () => await new StdSchedulerFactory(props).GetScheduler());

My workaround with a delay to allow jobs to terminate below works, but is so dodgy - kindly advise how I can get this working properly without an brittle arbitrary delay:

public async Task StopAsync(CancellationToken cancellationToken)
{
    await this.scheduler.PauseAll(cancellationToken);
    foreach (var job in await this.scheduler.GetCurrentlyExecutingJobs(cancellationToken))
    {
        this.logger.LogInformation($"Interrupting job {job.JobDetail}");
        await this.scheduler.Interrupt(job.JobDetail.Key, cancellationToken);

    }
    await Task.Delay(3000);
    await this.scheduler.Shutdown(cancellationToken);
}
like image 940
morleyc Avatar asked Jun 27 '21 17:06

morleyc


People also ask

How do you stop a job in quartz scheduler?

deleteJob(jobKey(<JobKey>, <JobGroup>)); This method will only interrupt/stop the job uniquely identified by the Job Key and Group within the scheduler which may have many other jobs running. scheduler. shutdown();

Why we should not use Quartz scheduler?

It has no built-in UI for configuration or monitoring, no useful and reasonably searchable logging or historical review, no support for multiple execution nodes, no administration interface, no alerts or notifications, not to mention buggy recovery mechanisms for failed jobs and missed jobs.

How does Quartz Scheduler work internally?

Quartz scheduler allows an enterprise to schedule a job at a specified date and time. It allows us to perform the operations to schedule or unschedule the jobs. It provides operations to start or stop or pause the scheduler. It also provides reminder services.

How do you schedule multiple jobs using Quartz?

If you want to schedule multiple jobs in your console application you can simply call Scheduler. ScheduleJob (IScheduler) passing the job and the trigger you've previously created: IJobDetail firstJob = JobBuilder. Create<FirstJob>() .

Can scheduler trigger new instances when jobs are stopped?

Sign in to your account The scheduler should never goes into situation where jobs are stopped triggerring new instances without raising any exception. We have any the quartz in production from last 3 years but in last one year we are frequently getting situation where jobs went in

What is the life-cycle of a scheduler?

A Scheduler 's life-cycle is bounded by it's creation, via a SchedulerFactory and a call to its Shutdown () method. Once created the IScheduler interface can be used add, remove, and list Jobs and Triggers, and perform other scheduling-related operations (such as pausing a trigger).

How do I schedule a job in a trigger?

When you wish to schedule a job, you instantiate a trigger and 'tune' its properties to provide the scheduling you wish to have. Triggers may also have a JobDataMap associated with them - this is useful to passing parameters to a Job that are specific to the firings of the trigger.


Video Answer


1 Answers

If you check the source code of generic host, you'll find that on host shutdown it waits for a default shutdown timeout, which is 5 seconds. It means that if your jobs take more time to complete, the host will exit by timeout and so will the application.

In addition, based on your comment, the scheduler has to be configured to interrupt the running jobs on shutdown:

var props = new NameValueCollection
{
    {"quartz.scheduler.interruptJobsOnShutdownWithWait", "true"},
};

var scheduler = AsyncContext.Run(async () => await new StdSchedulerFactory(props).GetScheduler());

and invoked with the waitForJobsToComplete parameter set to true for shutdown:

await this.scheduler.Shutdown(waitForJobsToComplete: true, cancellationToken);

to ensure that scheduler only exits when all jobs are completed.

To guarantee that the application exits only after all jobs are interrupted and completed, you could initiate shutdown after the host has exited:

public static Task Main(string[] args)
{
    using (var host = CreateHostBuilder(args)
        .UseSerilog<Settings>(Settings.Name)
        .UseConsoleLifetime()
        .Build()
        .UseSimpleInjector(container))
    {
        var logger = container.GetInstance<ILogger<Program>>();

        try
        {
            await host.RunAsync();

            var scheduller = container.GetInstance<IScheduler<Program>>();
            scheduller.Shutdown(true);
        }
        catch (Exception ex)
        {
            logger.LogCritical(ex, ex.Message);
        }

        logger.LogDebug($"Finish {nameof(Main)}().");
    }
}
like image 61
Andrii Litvinov Avatar answered Nov 06 '22 15:11

Andrii Litvinov