Sitecore EventQueue Table growing out of control

Question

We are having an issue with the EventQueue table growing very fast at times, up to 3k records a second, and never clearing records (30 million as of right now). Our environment has the following set up:

Sitecore 7.2

• 4 CD servers and 1 CM server
• All four CD servers are load balanced.
• CD1 and CD2 are pointed to DB1 server CD3 and CD4 are pointed to DB2
• server There are 2 Publishing targets (one for each DB) Merge
• Replication is setup for the Core db across all servers (CM, CD's)
• EventQueue is enabled

I have a few questions so I will break them down into separate line items.

1. When a publish is issued for all CD servers is the updated content sent directly from the CM db to the CD db's (all of the correct tables) or is it sent to the EventQueue table in the CD db and the CD server has a job/task that looks at the table and updates as needed.

2. Depending on answer to the first question, if there are 2 CD servers pointing to the same DB how do they know if they should process the EventQueue table (wont they each process the table and be duplicating efforts)

3. Why isn't the EventTable table cleared? How is is cleared, when is it cleared?

Answer 1

1. On CM publish, the publish request is sent to the EventQueue table on the CD db where it is processed as per the instance's publishing schedule.
2. The InstanceName column in the EventQueue table stores the unique name of each Sitecore instance (by default this is Machine Name + IIS Instance Name, but can be set in web.config). This enables events to be picked up by an individual CD instance in a load balanced environment.

3. The EventQueue table is cleared by a Sitecore task defined in the <scheduling> element in the web.config, although I've seen this misbehave in the past. By default, it is set as follows:

   <agent type="Sitecore.Tasks.CleanupEventQueue, Sitecore.Kernel" method="Run" interval="04:00:00"> <DaysToKeep>1</DaysToKeep> </agent>

I've previously run into high loads on the EventQueue and PublishQueue tables and would recommend trying the following (some of which were suggested from Sitecore support):

• Reduce the interval of the CleanupEventQueue agent (above)
• Reduce the DaysToKeep setting on the CleanupEventQueue (also CleanupPublishQueue wouldn't hurt)
• Create a scheduled SQL job to run the clean up script outlined in the CMS Tuning Guide (Page 10: http://sdn.sitecore.net/upload/sitecore7/70/cms_tuning_guide_sc70-usletter.pdf)

Finally, from Sitecore support:

Sitecore recommends that the number of rows (entries) in the History, PublishQueue, and EventQueue tables would be less than 1000.