Multiple region caches with Doctrine 2 second level cache and Symfony 3.3

Question

Have a distributed SF3.3 application running on multiple AWS EC2 instances with a central ElastiCache (redis) cluster.

Each EC2 instance also runs a local Redis instance which is used for Doctrine meta and query caching.

This application utilises Doctrines Second Level Cache, which works very well from a functional point of view. But performance is poor (900-1200ms page loads) on AWS due to the 400+ cache calls it makes to load in our Country and VatRate entities required on many of our pages.

As these Country and VatRate entities change rarely I'd like to utilise both the local Redis instance and ElastiCache for result caching by using different regions defined in the second level cache. This should reduce the latency problem with the 400+ cache calls as when running on a single box page loads are sub 100ms. Reading the documentation this all seems to be possible, just not entirely sure how to configure it with Symfony and PHP-Cache.

An example of the current configuration:

app/config/config.yml

doctrine:
    dbal:
        # .. params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    region_cache_driver:
                        type: service
                        id: doctrine.orm.default_result_cache

cache_adapter:
    providers:
        meta: # Used for version specific
            factory: 'cache.factory.redis'
            options:
                host: 'localhost'
                port: '%redis_local.port%'
                pool_namespace: "meta_%hash%"
        result: # Used for result data
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: result

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        result:
            service_id:         'cache.provider.result'
            entity_managers:    [ default ]

src/AppBundle/Entity/Country.php

/**
 * @ORM\Table(name = "countries")
 * @ORM\Cache(usage = "READ_ONLY")
 */
class Country
{
    // ...

    /**
     * @var VatRate
     *
     * @ORM\OneToMany(targetEntity = "VatRate", mappedBy = "country")
     * @ORM\Cache("NONSTRICT_READ_WRITE")
     */
    private $vatRates;

    // ...
}

src/AppBundle/Entity/VatRate.php

/**
 * @ORM\Table(name = "vatRates")
 * @ORM\Cache(usage = "READ_ONLY")
 */
class VatRate
{
    // ...

    /**
     * @var Country
     *
     * @ORM\ManyToOne(targetEntity = "Country", inversedBy = "vatRates")
     * @ORM\JoinColumn(name = "countryId", referencedColumnName = "countryId")
     */
    private $country;

    // ...
}

src/AppBundle/Entity/Order.php

/**
 * @ORM\Table(name = "orders")
 * @ORM\Cache(usage = "NONSTRICT_READ_WRITE")
 */
class Order
{
    // ...

    /**
     * @var Country
     *
     * @ORM\ManyToOne(targetEntity = "Country")
     * @ORM\JoinColumn(name = "countryId", referencedColumnName = "countryId")
     */
    private $country;
    // ...
}

Attempted Configuration

app/config/config.yml

doctrine:
    dbal:
        # .. params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    region_cache_driver: array
                    regions:
                        local:
                            type: service
                            service: "doctrine.orm.default_result_cache" # TODO: needs to be local redis 
                        remote:
                            type: service
                            service: "doctrine.orm.default_result_cache" # TODO: needs to be remote redis

cache_adapter:
    providers:
        meta: # Used for version specific
            factory: 'cache.factory.redis'
            options:
                host: 'localhost'
                port: '%redis_local.port%'
                pool_namespace: "meta_%hash%"
        result: # Used for result data
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: result

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.meta'
            entity_managers:    [ default ]
        result:
            service_id:         'cache.provider.result'
            entity_managers:    [ default ]

src/AppBundle/Entity/Country.php

/**
 * @ORM\Table(name = "countries")
 * @ORM\Cache(usage = "READ_ONLY", region = "local")
 */
class Country
{
    // as above
}

src/AppBundle/Entity/VatRate.php

/**
 * @ORM\Table(name = "vatRates")
 * @ORM\Cache(usage = "READ_ONLY", region = "local")
 */
class VatRate
{
    // as above
}

src/AppBundle/Entity/Order.php

/**
 * @ORM\Table(name = "orders")
 * @ORM\Cache(usage = "NONSTRICT_READ_WRITE", region = "remote")
 */
class Order
{
    // as above
}

Which results in

Type error: Argument 1 passed to Doctrine\ORM\Cache\DefaultCacheFactory::setRegion() must be an instance of Doctrine\ORM\Cache\Region, instance of Cache\Bridge\Doctrine\DoctrineCacheBridge given,

Not too sure where to go from here, been working from the tests here: https://github.com/doctrine/DoctrineBundle/blob/74b408d0b6b06b9758a4d29116d42f5bfd83daf0/Tests/DependencyInjection/Fixtures/config/yml/orm_second_level_cache.yml but the lack of documentation for configuring this makes it a little more challenging!

Answer 1

After much playing around with the PHP-Cache library, it's clear from looking in the CacheBundle compiler that it will only ever support one DoctrineBridge instance from the configuration. https://github.com/php-cache/cache-bundle/blob/master/src/DependencyInjection/Compiler/DoctrineCompilerPass.php

Solution was to create my own compiler, not pretty but it seems to work.

src/AppBundle/DependencyInjection/Compiler/DoctrineCompilerPass.php

namespace AppBundle\DependencyInjection\Compiler;

use Cache\Bridge\Doctrine\DoctrineCacheBridge;
use Cache\CacheBundle\Factory\DoctrineBridgeFactory;
use Symfony\Component\DependencyInjection\Compiler\CompilerPassInterface;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\DependencyInjection\Reference;

class DoctrineCompilerPass implements CompilerPassInterface
{
    /** @var ContainerBuilder */
    private $container;

    public function process(ContainerBuilder $container)
    {
        $this->container = $container;

        $this->enableDoctrineCache('local');
        $this->enableDoctrineCache('remote');
    }

    private function enableDoctrineCache(string $configName)
    {
        $typeConfig = [
            'entity_managers' => [
                'default'
            ],
            'use_tagging' => true,
            'service_id' => 'cache.provider.' . $configName
        ];

        $bridgeServiceId = sprintf('cache.service.doctrine.%s.entity_managers.bridge', $configName);

        $this->container->register($bridgeServiceId, DoctrineCacheBridge::class)
            ->setFactory([DoctrineBridgeFactory::class, 'get'])
            ->addArgument(new Reference($typeConfig['service_id']))
            ->addArgument($typeConfig)
            ->addArgument(['doctrine', $configName]);
    }
}

src/AppBundle/AppBundle.php

use AppBundle\DependencyInjection\Compiler\DoctrineCompilerPass;
use Symfony\Component\DependencyInjection\ContainerBuilder;
use Symfony\Component\HttpKernel\Bundle\Bundle;

class AppBundle extends Bundle
{
    public function build(ContainerBuilder $container)
    {
        parent::build($container);

        $container->addCompilerPass(new DoctrineCompilerPass());
    }
}

app/config/config.yml

doctrine:
    dbal:
    # ... params

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        entity_managers:
            default:
                auto_mapping: true
                second_level_cache:
                    enabled: true
                    regions:
                        remote:
                            cache_driver:
                                type: service
                                id: cache.service.doctrine.remote.entity_managers.bridge
                        local:
                            cache_driver:
                                type: service
                                id: cache.service.doctrine.local.entity_managers.bridge

cache_adapter:
    providers:
        local:
            factory: 'cache.factory.redis'
            options:
                host: '%redis_local.host%'
                port: '%redis_local.port%'
                pool_namespace: "local_%hash%"
        remote:
            factory: 'cache.factory.redis'
            options:
                host: '%redis_result.host%'
                port: '%redis_result.port%'
                pool_namespace: 'result'

cache:
    doctrine:
        enabled: true
        use_tagging: true
        metadata:
            service_id:         'cache.provider.local'
            entity_managers:    [ default ]
        query:
            service_id:         'cache.provider.local'
            entity_managers:    [ default ]

While this seems to work to some extent, there's some inconsistencies local cache calls resulting in 500 errors when theres probably something missing in the cache. Overall think I'm trying to bend the second level cache more than it was designed to.