How to unit test Symfony validation constraints?

Question

What is the best way to verify that a class has all the expected constraints (property constraints, get/is/has method constraints, custom callback constraints, class constraints) set?

A good way would be a way that:

• allows defining set of constraints expected to be set on given class and is able to identify missing ones
• doesn't require running actual validation code, since this code is already tested by the validator class test
• has reasonable "arrange" step (e.g. the setUp method) which is fairly easy to prepare and understandable for average test user

We've already tried different solutions (see 1. and 2. in Code examples) but none of them really solves this problem.

Idea with verifying metadata (see 3. in Code examples) seems promising, but maybe someone can come up with something better?

Use cases

To give some real-life usages:

1. There is a Foo entity class and a Bar entity class that extends Foo.

   Someone removes NotNull constraint from one of Foo's fields (updating Foo's unit test before).

   We want to make sure that we have a suite of unit tests that will detect that Bar doesn't have NotNull constraint on it's field anymore.

2. Let's assume there is a validator which implements quite complicated validation logic, e.g. "given date has to be on 1st Monday of the month, unless it's June or October, then it has to be last Tuesday, but then ..." (you know what I mean).
   This validator is fully unit tested and we know it works as we want.

   Let's also assume there are 100 entity classes with this constraint set on some fields.

   The process of setting up object to test is complicated (it's already done in validator unit test). We don't want to do it 100 times more.
   And if one day requirements for this validation change, e.g. the "Tuesday" condition has to extended to "Tuesday or Wednesday but only if ...", then we shouldn't be forced to change 101 unit tests but just one - the unit test for validator class.

Code examples

Here are the code snippets that are referred to above:

1. Test constraints by actually running the validation process using validator service and counting created violations.

   class MyEntityValidationTest extends \PHPUnit_Framework_TestCase
   {
       public function testMyEntityIsInvalidBecauseOfSomething()
       {
           $containerProphecy = $this->prophesize(ContainerInterface::class);
   
           $validator = Validation::createValidatorBuilder()
               ->enableAnnotationMapping()
               ->setConstraintValidatorFactory(new ConstraintValidatorFactory($containerProphecy->reveal(), []))
               ->getValidator();
   
           $myEntity = new MyEntity();
           $violations = $validator->validate($myEntity);
   
           $this->assertCount(1, $violations);
       }
   
       public function testMyEntityIsInvalidBecauseOfSomethingElse()
       {
           // if you use validator.constraint_validator tag with an alias for your validator service (the old way),
           // you also have to set up get() method call on container mock and define mapping
   
           $containerProphecy = $this->prophesize(ContainerInterface::class);
           $containerProphecy->get('my_entity_validator_service')->willReturn(new MyEntityValidator());
   
           $validators = [
               'my_entity_validator_service' => 'validator.my_entity',
           ];
   
           $validator = Validation::createValidatorBuilder()
               ->enableAnnotationMapping()
               ->setConstraintValidatorFactory(new ConstraintValidatorFactory($containerProphecy->reveal(), $validators))
               ->getValidator();
   
           $myEntity = new MyEntity();
           $violations = $validator->validate($myEntity);
   
           $this->assertCount(1, $violations);
       }
   }
   

   The main problem with this solution is that:

   • it tests actual implementation of validator which is/should be already tested in validator class test (so technically it's not unit test);
   • the setup step, i.e. creating validator, defining container mock and map of validators, can be quite complex and makes test hard to read.

2. Test constraints by mocking the context object passed to validator->initialize() method and setting up all the expectations needed to make the test run.

   class MyEntityValidationTest extends \PHPUnit_Framework_TestCase
   {
       public function testMyEntityIsInvalidBecauseOfSomething()
       {
           $myEntity = $this->prophesize(MyEntity::class);
           $myEntity
               ->isStillValid()
               ->willReturn(false);
   
           $constraintProphecy = $this->prophesize(MyEntityConstraint::class);
   
           $validator = new MyEntityValidator();
           $validator->initialize($this->buildContextWithExpectedViolation());
   
           $validator->validate($myEntity->reveal(), $constraintProphecy->reveal());
       }
   
       private function buildContextWithExpectedViolation()
       {
           $violationBuilderProphecy = $this->prophesize(ConstraintViolationBuilder::class);
           $violationBuilderProphecy
               ->atPath('configurationId')
               ->shouldBeCalled()
               ->willReturn($violationBuilderProphecy);
           // if you want to use translation messages with parameters you need to configure it as well
           $violationBuilderProphecy
               ->setParameter('%configuration_id%', Argument::any())
               ->shouldBeCalled()
               ->willReturn($violationBuilderProphecy);
           $violationBuilderProphecy
               ->addViolation()
               ->shouldBeCalled();
   
           $contextProphecy = $this->prophesize(ExecutionContextInterface::class);
           $contextProphecy
               ->buildViolation('error message')
               ->willReturn($violationBuilderProphecy->reveal());
   
           return $contextProphecy->reveal();
       }
   }
   

   Again, the problem with this solution is the setup step, i.e. preparing all mocks and defining expectations can be quite difficult for both, the creator and also for the reader of the test.

   Symfony (technically, Validator component) provides a helper TestCase class (Symfony\Component\Validator\Test\ConstraintValidatorTestCase) that can be used to make writing test like this a bit easier, but still it's not perfect.

3. Test constraints by preparing the validator service, getting metadata information generated for a class you want to test and verify that all expected constraints are defined.

   class MyEntityValidationTest extends \PHPUnit_Framework_TestCase
   {
       public function testMyEntityHasExpectedValidationConstraintsAttached()
       {
           $classMetadata = Validation::createValidatorBuilder()
               ->enableAnnotationMapping()
               ->getValidator()
               ->getMetadataFor(new MyEntity());
   
           $this->assertClassContainsConstraint($classMetadata, FooConstraint::class);
           $this->assertClassContainsConstraint($classMetadata, BarConstraint::class, [
               'thisField' => 123,
               'thatField' => 'test',
           ]);
   
           $namePropertyMetadata = $classMetadata->getPropertyMetadata('name');
           $this->assertPropertyContainsConstraint($namePropertyMetadata, NotBlank::class);
   
           $valuePropertyMetadata = $classMetadata->getPropertyMetadata('value');
           $this->assertPropertyContainsConstraint($valuePropertyMetadata, GreaterThan::class, [
               'value' => 100,
           ]);
       }
   
       private function assertClassContainsConstraint(
           MetadataInterface $classMetadata,
           string $constraintClass,
           array $constraintFields = []
       ) {
           // $this->fail() if none of constrains in $classMetadata
           // matches $constraintClass / $constraintFields
       }
   
       private function assertPropertyContainsConstraint(
           PropertyMetadataInterface $propertyMetadata,
           string $constraintClass,
           array $constraintFields = []
       ) {
           // $this->fail() if none of constrains in $propertyMetadata
           // matches $constraintClass / $constraintFields
       }
   }
   

   This needs some extra work to implement assertClassContainsConstraint(), assertPropertyContainsConstraint() or similar methods, but once done (e.g. in abstract TestCase class) it can be reused in all other tests classes.

   Also, this has the added benefit of handling constraints defined in different format, e.g. annotations, yaml, xml, php.

   Note that this uses PHPUnit's assertSomething() convention. It would be even better to be able to use phpspec's "specking" approach, to be able to tell what is expected to happen while preparing validator service.

Answer 1

As you said, the Validator should be already tested, so you don't need to test a validation works.

I think you want to test only that your assertions work depending your business logic.

allows defining set of constraints expected to be set on given class and is able to identify missing ones

If you create a set of constraints that you use on multiple entities, then you should unit test each of these entities, to check if violations are created, no matter the code behind.

doesn't require running actual validation code, since this code is already tested by the validator class test

You should not think about code, but only your business logic. You need a test for each use case.

For that, your first example is near the solution. Here is how I test that I validate well my class:

<?php

namespace App\Tests\Validator;

use App\Validator\NicknameValidator;
use PHPUnit\Framework\TestCase;
use Symfony\Component\Validator\Constraint;
use Symfony\Component\Validator\Context\ExecutionContext;
use Symfony\Component\Validator\Validator\ValidatorInterface;
use Symfony\Contracts\Translation\TranslatorInterface;

class NicknameValidatorTest extends TestCase
{
    public function testValidateDetectInvalidNickname(): void
    {
        $nicknameValidator = new NicknameValidator();

        $context = new ExecutionContext(
            $this->prophesize(ValidatorInterface::class)->reveal(),
            null,
            $this->prophesize(TranslatorInterface::class)->reveal()
        );

        $context->setConstraint($this->prophesize(Constraint::class)->reveal());

        $nicknameValidator->initialize($context);
        $nicknameValidator->validate('Test%', $this->prophesize(Constraint::class)->reveal());

        $this->assertEquals(
            'Nickname cannot contains special characters.',
            $context->getViolations()->get(0)->getMessageTemplate()
        );
    }
}

The simplest solution is often the best one.