Why Does Magento Invalidation of Full Page Caching on Product save actually make a page not cached and what does refresh do since it is not cached

Question

So Caching of course is what confuses me the most in Magento, as it does for most others I am sure. Currently one of the sites we work on is on Enterprise and utilizes FPC of course. The problem is, we have an inventory update that runs every 15 minutes. A lot of orders are placed to CSR's over the phone and through a catalog into an external system outside of Magento.

Every 15 minutes a script is ran to check any inventory in that system and to see if it differs then what is in Magento. If there is a difference then the inventory is updated in Magento. Using all Magento methods, no sql or anything like that.

We have always had caching issues and have tried all of the latest techniques when they come out. The latest one we are trying is Redis, and we have had good success on other sites with it. However, we are still seeing crazy load on the server and it is apparent that pages aren't cached.

After digging into the code it appears that after ever every model save or admin product controller save it looks to see if cache needs to be invalidated. It appears that changing any attribute, well at least inventory will mark FPC as needing to be invalidated.

I am confused about what invalidation means, because a while back we had a question out to customer support about something similar and this was the response

Full Page cache will get to invalidated state upon any changes on the products, categories, CMS even when the stock is decreased after a sale.

Now when full page cache gets to invalidated state this does not mean that something is changed on your frontend however any changes applied after the last refresh will not be shown on the frontend.

However if having the FPC validated at all times is a must for your business logic you could certainly set your Magento Installation to refresh it automatically through cron functionality as often as you desire.

However on all of the tests that I have done, on both 1.9 and 1.11 Enterprise, it appears when FPC is invalidated, the response isn't being pulled in from cache. Which is contradictory to what they have said about it just not having the newer updates.

Is there something I am missing? Does anyone have a good explanation for how the invalidation works in Magento specifically for FPC or any good links to fully understand the process and the code?

You can try this yourself for any page that is full page cached. But it is my understanding that the method processRequest in /app/code/core/Mage/Core/Model/Cache.php should set the body content with the cached response and return true if the page is cached.

To test go to any page make sure you got it cached and returning true. Go in and edit a product, in our case quantity. This will invalidate FPC. However now when you load the page that was cached before it will return false in this method and not be a cached page. I don't know if this is accurate to be able to tell if a page is cached or not but that is where my investigation lead me. Please correct me if I am wrong.

UPDATE: Upon further investigation I have found that when you save a product in the admin, the controller action

Mage_Adminhtml_Catalog_ProductController::saveAction()

will call the following method

Mage::getModel('catalogrule/rule')->applyAllRulesToProduct($productId)

Then in the Mage_CatalogRule_Model_Resource_Rule class, the applyAllRulesForDateRange method is called and that fires off the event

catalogrule_after_apply

Which the Full Page Cache module is observing and firing the clean cache method for the FPC tag. Essentially deleting all FPC cache records.

I don't see why this is necessary if previous to this the logic is clearing the FPC records that are tied to the product and category tags. Is this a bug?

Answer 1

FPC is observing inventory changes because the intent is to display out of stock for any products that have been decremented to zero stock. The fix would be to create an event dispatch when a product hits zero rather than every time any product changes stock and rewrite FPC to observe that event instead of the original.

Another method would be to only invalidate the portions of cache pertaining to the products being updated, but this would be a rather significant architectural change.

Answer 2

You shall create a new custom indexer Mage_Index_Model_Indexer_Abstract , and create a new resource model api methods with cron jobs