Quartz.NET with AdoNetJobStore performance

Question

I have ~5M Jobs and each Job has exactly one Trigger scheduled in Quartz.Net, as a maximum ~300K Jobs scheduled to run at same time, I have a constraint to proceed all 300K Jobs within 3 hours (so ~100K Jobs/Hour), but now my test app is able to proceed only 10K per hour when Quartz.Net is configured to use AdoNetJobStore.

I'm using next Quartz config:

<quartz>
    <add key="quartz.scheduler.instanceName" value="XxxDefaultQuartzScheduler" />
    <add key="quartz.scheduler.instanceId" value="instance_one" />
    <add key="quartz.threadPool.type" value="Quartz.Simpl.SimpleThreadPool, Quartz" />
    <add key="quartz.threadPool.threadCount" value="10" />
    <add key="quartz.threadPool.threadPriority" value="1" />
    <add key="quartz.jobStore.type" value="Quartz.Impl.AdoJobStore.JobStoreTX, Quartz" />
    <add key="quartz.jobStore.misfireThreshold" value="60000" />
    <add key="quartz.jobStore.dataSource" value="default" />
    <add key="quartz.jobStore.driverDelegateType" value="Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz" />
    <add key="quartz.jobStore.lockHandler.type" value="Quartz.Impl.AdoJobStore.UpdateLockRowSemaphore, Quartz" />
    <add key="quartz.jobStore.tablePrefix" value="QRTZ_" />
    <add key="quartz.jobStore.useProperties" value="false" />
    <add key="quartz.dataSource.default.connectionStringName" value="QuartzDbContext" />
    <add key="quartz.dataSource.default.provider" value="SqlServer-20" />
</quartz>

Is it possible to configure Quartz.Net with SQL Job Store to provide such performance?

Answer 1

The default thread count of 10 is quite low for this kind of massive usage. I'd increment it to something like, say, 25 x cores in computer. You can naturally test higher values too. Modern hardware should be able to handle 100-200 threads, but thread contention may come as an issue.

You didn't mention the version you are using, but with newer versions you can drop the quartz.jobStore.lockHandler.type , it's use optimized row locking when it detects that SQL Server is used. Always use the latest version (when possible), latest tends to be most optimized and bug-free.

After Quartz configuration is tweaked, you naturally must profile your own job code. Say your code would be able to complete in 5 seconds so what we would approximately have is:

5 seconds * 300 000 triggers / 100 threads => 15 000 seconds ~ 250 minutes ~ 4 hours

Lower that five seconds of execution time to three and you'll be hitting the 2h 30m mark. Or just raise the amount of threads and be prepared for some battle of resources - depends on what your jobs are doing and how.

Another consideration could be clustering, run multiple workers against Quartz database if the database isn't the bottleneck.