Quartz.NET scheduler doesn't fire jobs/triggers once deployed

Question

INTRODUCTION

I'm using Quartz.Net on an ASP.Net framework 4, webforms web site. Basically, the user should have the hability to fire manually a batch script that asynchronously processes thousands of records stored on a database. The user can stop or pause at any time, adjust some variables, and continue if needed with the process (remaining records).
The code is done and working locally (developer machine, win7, vs2010, sql server express 2008 R2).
It was also tested on a local server (win server 2008 R2, sql server express 2008 R2). It works fine on both enviroments, tested with all the code precompiled. The problem is that, once deployed on a remote server (win server 2008 R2), where it actually should be running on (hosting enviroment, not shared, not clustered), it doesnt completely works (se details below). Scheduler gets created, but the trigger, hence the job, doesn't fire.

(Note: I know some of you would suggest to use Quartz as windows service, but despite the benefits of doing so, I really would like to find out why it doesn't work as an embedded solution, since it should be working just fine like does locally)

DETAILS

Quartz 2.1.2  
Common.Logging 2.1.2  
Common.Logging.NLog 2.0.0  
NLog 2.0.1.2

global.asax

public static ISchedulerFactory SchedulerFactory;
public static IScheduler Scheduler;

void Application_Start(object sender, EventArgs e)
{
    SchedulerFactory = new StdSchedulerFactory();
    Scheduler = SchedulerFactory.GetScheduler();

    // Define a durable job instance (durable jobs can exist without triggers)
    IJobDetail job = JobBuilder.Create<MyJobClass>()
                                .WithIdentity("MyJob", "MyGroup")
                                .StoreDurably()
                                .Build();

    Scheduler.AddJob(job, false);
    Scheduler.Start();
}
void Application_End(object sender, EventArgs e)
{
    Scheduler.Shutdown(true);
}

process.aspx.cs (start button click)

// get records from DB, iterate, process, etc
...

IJobDetail job = ASP.global_asax.Scheduler.GetJobDetail(new JobKey("MyJob", "MyGroup"));
job.JobDataMap.Put("something1", 1);
job.JobDataMap.Put("something2", somevar);

ITrigger trigger = TriggerBuilder.Create()
                    .WithIdentity("MyTrigger", "MyGroup")
                    .StartNow()
                    .WithSimpleSchedule(x => x.WithIntervalInSeconds(5).RepeatForever())
                    .Build();

var triggersSet = new Quartz.Collection.HashSet<ITrigger> { trigger };

ASP.global_asax.Scheduler.ScheduleJob(job, triggersSet, true);

LOG OUTPUT

local log

Default Quartz.NET properties loaded from embedded resource file  
Using default implementation for object serializer  
Using default implementation for ThreadExecutor  
Initialized Scheduler Signaller of type: Quartz.Core.SchedulerSignalerImpl  
Quartz Scheduler v.2.1.2.400 created.  
RAMJobStore initialized.  
Scheduler meta-data: Quartz Scheduler (v2.1.2.400) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED'   Scheduler class: 'Quartz.Core.QuartzScheduler' - running locally.   NOT STARTED.   Currently in standby mode.   Number of jobs executed: 0   Using thread pool 'Quartz.Simpl.SimpleThreadPool' - with 10 threads.   Using job-store 'Quartz.Simpl.RAMJobStore' - which does not support persistence. and is not clustered.  
Quartz scheduler 'DefaultQuartzScheduler' initialized  
Quartz scheduler version: 2.1.2.400  
Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.  
Batch acquisition of 0 triggers  
Batch acquisition of 0 triggers

It continues logging Batch acquisition of 0 triggers until button click occurs:

Default Quartz.NET properties loaded from embedded resource file  
Batch acquisition of 1 triggers  
Producing instance of Job 'MyGroup.MyJob', class=MyJobClass  
Batch acquisition of 0 triggers  
Calling Execute on job MyGroup.MyJob  
Trigger instruction : NoInstruction  
Batch acquisition of 1 triggers  
Producing instance of Job 'MyGroup.MyJob', class=MyJobClass  
Batch acquisition of 0 triggers  
Calling Execute on job MyGroup.MyJob  
Trigger instruction : NoInstruction  
Batch acquisition of 1 triggers

deployed log

Default Quartz.NET properties loaded from embedded resource file  
Using default implementation for object serializer  
Using default implementation for ThreadExecutor  
Initialized Scheduler Signaller of type: Quartz.Core.SchedulerSignalerImpl  
Quartz Scheduler v.2.1.2.400 created.  
RAMJobStore initialized.  
Scheduler meta-data: Quartz Scheduler (v2.1.2.400) 'DefaultQuartzScheduler' with instanceId 'NON_CLUSTERED' Scheduler class: 'Quartz.Core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0 Using thread pool 'Quartz.Simpl.SimpleThreadPool' - with 10 threads. Using job-store 'Quartz.Simpl.RAMJobStore' - which does not support persistence. and is not clustered.   
Quartz scheduler 'DefaultQuartzScheduler' initialized  
Quartz scheduler version: 2.1.2.400  
Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.

Here stays like this. As you see, compared to the other log, it's not trying to acquire triggers (line Batch acquisition of 0 triggers does not appear at all). If you click the process button anyway, the log adds one line:

Default Quartz.NET properties loaded from embedded resource file

But nothing else happens. The records are not processed (I know since every time a record is proccessed, is marked in the database). No errors occur, but the trigger is not fired, and the job is not executed. Also, the CPU usage run up to 50% or more on button click, and doesnt gets down unless you go to IIS, stop and restart the application pool. This cpu consumption doesn't happen locally.

---

update 1

Changed use of scheduler for a singleton, as suggested by LeftyX, but still get same behavior on remote server.

update 2

I also tried to use ADOJobStore (instead of RAMJobStore which I was using). Now it still works perfectly locally; but still doesn't execute the trigger (hence job) online. The only difference is that online the CPU usage doesn't run up to 50%. And now I can see that the job and trigger are created (I query the tables and see that those records exists), but never gets executed.

Answer 1

There are nothing wrong with Quartz, all because of IIS app pool recycling. I fixed the bug by stopping the pool that is used for Quartz from recycling:

1. Go to IIS manager -> Application Pools -> Create a new pool, I named it Scheduler (any name is ok)
   
2. Select Scheduler pool -> advanced Settings
   

• In General section, at Start Mode, Select AlwaysRunning (IIS 8.5) or true for (IIS 7.5, 8)
  
• In Process Model Section-> Idle Timeout(minutes) set to 0 (meaning: No Idel timeout)
  
• In Recycling section -> Regular time Interval set to 0 (meaning: no recycling)
  3. Deploy your Quartz site into that application pool. And send one request to the pool to "wake your app up" and it will run until u stop it.
  That's it.
  Update: Another solution to keep your app pool always alive is using Auto-Start ASP.NET Applications

Another option: Using some third party pinging tool (like uptimerobot or diy one ) to keep refreshing your site each and every x seconds (or minute)