How to create a twig custom tag that executes a callback?

Question

I'm trying to create a custom Twig tag such as this:

{% mytag 'foo','bar' %}
   Hello world!!
{% endmytag %}

This tag should print the output of my func("Hello world!!", "foo", "bar").

Can anybody post some sample code for creating such custom tag? One that may accept a arbitrary number of parameters would me even more appreciated.

note: I'm not interested in creating a custom function, I need to have the body of the tag passed in as the first parameter.

Answer 1

Theory

Before speaking about tags, you should understand how Twig works internally.

• First, as Twig code can be put on a file, on a string or even on a database, Twig opens and reads your stream using a Loader. Most known loaders are Twig_Loader_Filesystem to open twig code from a file, and Twig_Loader_Array to get twig code directly from a string.

• Then, this twig code is parsed to build a parse tree, containing an object representation of the twig code. Each object are called Node, because they are part of a tree. As other languages, Twig is made of tokens, such as {%, {#, function(), "string"... so Twig language constructs will read for several tokens to build the right node.

• The parse tree is then walked across, and compiled into PHP code. The generated PHP classes follow the Twig_Template interface, so the renderer can call the doDisplay method of that class to generate the final result.

If you enable caching, you can see those generated files and understand what's going on.

---

Let's start practicing smoothly...

All internal twig tags, such as {% block %}, {% set %}... are developed using the same interfaces as custom tags, so if you need some specific samples, you can look at Twig source code.

But, the sample you want is a good start anyway, so let's develop it.

The TokenParser

The token parser's goal is to parse and validate your tag arguments. For example, the {% macro %} tag requires a name, and will crash if you give a string instead.

When Twig finds a tag, it will look into all registered TokenParser classes the tag name returned by getTag() method. If the name match, then Twig calls the parse() method of that class.

When parse() is called, the stream pointer is still on the tag name token. So we should get all inline arguments, and finish the tag declaration by finding an BLOCK_END_TYPE token. Then, we subparse the tag's body (what is contained inside the tag, as it also may contain twig logic, such as tags and other stuffs): the decideMyTagFork method will be called each time a new tag is found in the body: and will break the sub parsing if it returns true. Note that this method name does not take part of the interface, that's just a standard used on Twig's built-in extensions.

For reference, Twig tokens can be the following:

• EOF_TYPE: last token of the stream, indicating the end.

• TEXT_TYPE: the text that does not take part of twig language: for example, in the Twig code Hello, {{ var }}, hello, is a TEXT_TYPE token.

• BLOCK_START_TYPE: the "begin to execute statement" token, {%

• VAR_START_TYPE: the "begin to get expression result" token, {{

• BLOCK_END_TYPE: the "finish to execute statement" token, %}

• VAR_END_TYPE: the "finish to get expression result" token, }}

• NAME_TYPE: this token is like a string without quotes, just like a variable name in twig, {{ i_am_a_name_type }}

• NUMBER_TYPE: nodes of this type contains number, such as 3, -2, 4.5...

• STRING_TYPE: contains a string encapsulated with quotes or doublequotes, such as 'foo' and "bar"

• OPERATOR_TYPE: contains an operator, such as +, -, but also ~, ?... You will about never need this token as Twig already provide an expression parser.

• INTERPOLATION_START_TYPE, the "begin interpolation" token (since Twig >= 1.5), interpolations are expressions interpretation inside twig strings, such as "my string, my #{variable} and 1+1 = #{1+1}". Beginning of the interpolation is #{.

• INTERPOLATION_END_TYPE, the "end interpolation" token (since Twig >= 1.5), unescaped } inside a string when an interpolation was open for instance.

MyTagTokenParser.php

<?php

class MyTagTokenParser extends \Twig_TokenParser
{

   public function parse(\Twig_Token $token)
   {
      $lineno = $token->getLine();

      $stream = $this->parser->getStream();

      // recovers all inline parameters close to your tag name
      $params = array_merge(array (), $this->getInlineParams($token));

      $continue = true;
      while ($continue)
      {
         // create subtree until the decideMyTagFork() callback returns true
         $body = $this->parser->subparse(array ($this, 'decideMyTagFork'));

         // I like to put a switch here, in case you need to add middle tags, such
         // as: {% mytag %}, {% nextmytag %}, {% endmytag %}.
         $tag = $stream->next()->getValue();

         switch ($tag)
         {
            case 'endmytag':
               $continue = false;
               break;
            default:
               throw new \Twig_Error_Syntax(sprintf('Unexpected end of template. Twig was looking for the following tags "endmytag" to close the "mytag" block started at line %d)', $lineno), -1);
         }

         // you want $body at the beginning of your arguments
         array_unshift($params, $body);

         // if your endmytag can also contains params, you can uncomment this line:
         // $params = array_merge($params, $this->getInlineParams($token));
         // and comment this one:
         $stream->expect(\Twig_Token::BLOCK_END_TYPE);
      }

      return new MyTagNode(new \Twig_Node($params), $lineno, $this->getTag());
   }

   /**
    * Recovers all tag parameters until we find a BLOCK_END_TYPE ( %} )
    *
    * @param \Twig_Token $token
    * @return array
    */
   protected function getInlineParams(\Twig_Token $token)
   {
      $stream = $this->parser->getStream();
      $params = array ();
      while (!$stream->test(\Twig_Token::BLOCK_END_TYPE))
      {
         $params[] = $this->parser->getExpressionParser()->parseExpression();
      }
      $stream->expect(\Twig_Token::BLOCK_END_TYPE);
      return $params;
   }

   /**
    * Callback called at each tag name when subparsing, must return
    * true when the expected end tag is reached.
    *
    * @param \Twig_Token $token
    * @return bool
    */
   public function decideMyTagFork(\Twig_Token $token)
   {
      return $token->test(array ('endmytag'));
   }

   /**
    * Your tag name: if the parsed tag match the one you put here, your parse()
    * method will be called.
    *
    * @return string
    */
   public function getTag()
   {
      return 'mytag';
   }

}

The compiler

The compiler is the code that will write in PHP what your tag should do. In your example, you want to call a function with body as first parameter, and all tag arguments as other parameters.

As the body entered between {% mytag %} and {% endmytag %} might be complex and also compile its own code, we should trick using output buffering (ob_start() / ob_get_clean()) to fill the functionToCall()'s argument.

MyTagNode.php

<?php

class MyTagNode extends \Twig_Node
{

   public function __construct($params, $lineno = 0, $tag = null)
   {
      parent::__construct(array ('params' => $params), array (), $lineno, $tag);
   }

   public function compile(\Twig_Compiler $compiler)
   {
      $count = count($this->getNode('params'));

      $compiler
         ->addDebugInfo($this);

      for ($i = 0; ($i < $count); $i++)
      {
         // argument is not an expression (such as, a \Twig_Node_Textbody)
         // we should trick with output buffering to get a valid argument to pass
         // to the functionToCall() function.
         if (!($this->getNode('params')->getNode($i) instanceof \Twig_Node_Expression))
         {
            $compiler
               ->write('ob_start();')
               ->raw(PHP_EOL);

            $compiler
               ->subcompile($this->getNode('params')->getNode($i));

            $compiler
               ->write('$_mytag[] = ob_get_clean();')
               ->raw(PHP_EOL);
         }
         else
         {
            $compiler
               ->write('$_mytag[] = ')
               ->subcompile($this->getNode('params')->getNode($i))
               ->raw(';')
               ->raw(PHP_EOL);
         }
      }

      $compiler
         ->write('call_user_func_array(')
         ->string('functionToCall')
         ->raw(', $_mytag);')
         ->raw(PHP_EOL);

      $compiler
         ->write('unset($_mytag);')
         ->raw(PHP_EOL);
   }

}

The extension

That's cleaner to create an extension to expose your TokenParser, because if your extension needs more, you'll declare everything's required here.

MyTagExtension.php

<?php

class MyTagExtension extends \Twig_Extension
{

   public function getTokenParsers()
   {
      return array (
              new MyTagTokenParser(),
      );
   }

   public function getName()
   {
      return 'mytag';
   }

}

Let's test it!

mytag.php

<?php

require_once(__DIR__ . '/Twig-1.15.1/lib/Twig/Autoloader.php');
Twig_Autoloader::register();

require_once("MyTagExtension.php");
require_once("MyTagTokenParser.php");
require_once("MyTagNode.php");

$loader = new Twig_Loader_Filesystem(__DIR__);

$twig = new Twig_Environment($loader, array (
// if you want to look at the generated code, uncomment this line
// and create the ./generated directory
//        'cache' => __DIR__ . '/generated',
   ));

function functionToCall()
{
   $params = func_get_args();
   $body = array_shift($params);
   echo "body = {$body}", PHP_EOL;
   echo "params = ", implode(', ', $params), PHP_EOL;
}


$twig->addExtension(new MyTagExtension());
echo $twig->render("mytag.twig", array('firstname' => 'alain'));

mytag.twig

{% mytag 1 "test" (2+3) firstname %}Hello, world!{% endmytag %}

Result

body = Hello, world!
params = 1, test, 5, alain

Going further

If you enable your cache, you can see the generated result:

protected function doDisplay(array $context, array $blocks = array())
{
    // line 1
    ob_start();
    echo "Hello, world!";
    $_mytag[] = ob_get_clean();
    $_mytag[] = 1;
    $_mytag[] = "test";
    $_mytag[] = (2 + 3);
    $_mytag[] = (isset($context["firstname"]) ? $context["firstname"] : null);
    call_user_func_array("functionToCall", $_mytag);
    unset($_mytag);
}

For this specific case, this will work even if you put others {% mytag %} inside a {% mytag %} (eg, {% mytag %}Hello, world!{% mytag %}foo bar{% endmytag %}{% endmytag %}). But if you're building such a tag, you will probably use more complex code, and overwrite your $_mytag variable by the fact it has the same name even if you're deeper in the parse tree.

So let's finish this sample by making it robust.

The NodeVisitor

A NodeVisitor is like a listener: when the compiler will read the parse tree to generate code, it will enter all registered NodeVisitor when entering or leaving a node.

So our goal is simple: when we enter a Node of type MyTagNode, we'll increment a deep counter, and when we leave a Node, we'll decrement this counter. In the compiler, we will be able to use this counter to generate the right variable name to use.

MyTagNodeVisitor.php

<?php

class MyTagNodevisitor implements \Twig_NodeVisitorInterface
{

   private $counter = 0;

   public function enterNode(\Twig_NodeInterface $node, \Twig_Environment $env)
   {
      if ($node instanceof MyTagNode)
      {
         $node->setAttribute('counter', $this->counter++);
      }
      return $node;
   }

   public function leaveNode(\Twig_NodeInterface $node, \Twig_Environment $env)
   {
      if ($node instanceof MyTagNode)
      {
         $node->setAttribute('counter', $this->counter--);
      }
      return $node;
   }

   public function getPriority()
   {
      return 0;
   }

}

Then register the NodeVisitor in your extension:

MyTagExtension.php

class MyTagExtension
{

    // ...
    public function getNodeVisitors()
    {
        return array (
                new MyTagNodeVisitor(),
        );
    }

}

In the compiler, replace all "$_mytag" by sprintf("$mytag[%d]", $this->getAttribute('counter')).

MyTagNode.php

  // ...
  // replace the compile() method by this one:

  public function compile(\Twig_Compiler $compiler)
   {
      $count = count($this->getNode('params'));

      $compiler
         ->addDebugInfo($this);

      for ($i = 0; ($i < $count); $i++)
      {
         // argument is not an expression (such as, a \Twig_Node_Textbody)
         // we should trick with output buffering to get a valid argument to pass
         // to the functionToCall() function.
         if (!($this->getNode('params')->getNode($i) instanceof \Twig_Node_Expression))
         {
            $compiler
               ->write('ob_start();')
               ->raw(PHP_EOL);

            $compiler
               ->subcompile($this->getNode('params')->getNode($i));

            $compiler
               ->write(sprintf('$_mytag[%d][] = ob_get_clean();', $this->getAttribute('counter')))
               ->raw(PHP_EOL);
         }
         else
         {
            $compiler
               ->write(sprintf('$_mytag[%d][] = ', $this->getAttribute('counter')))
               ->subcompile($this->getNode('params')->getNode($i))
               ->raw(';')
               ->raw(PHP_EOL);
         }
      }

      $compiler
         ->write('call_user_func_array(')
         ->string('functionToCall')
         ->raw(sprintf(', $_mytag[%d]);', $this->getAttribute('counter')))
         ->raw(PHP_EOL);

      $compiler
         ->write(sprintf('unset($_mytag[%d]);', $this->getAttribute('counter')))
         ->raw(PHP_EOL);
   }

Don't forget to include the NodeVisitor inside the sample:

mytag.php

// ...
require_once("MyTagNodeVisitor.php");

Conclusion

Custom tags are a very powerful way to extend twig, and this introduction gives you a good start. There are lots of features not described here, but by looking close to twig built-in extensions, abstract classes extended by the classes we written, and moreover by reading the generated php code resulting from twig files, you'll get everything to create any tag you want.

Download this sample

Answer 2

After looking at documentation.. Not sure that it follows all the standards, but it works..

require 'Twig/Autoloader.php';
Twig_AutoLoader::register();

class MyTag_TokenParser extends Twig_TokenParser
{
    public function parse(Twig_Token $token)
    {
        $parser = $this->parser;
        $stream = $parser->getStream();

        if (!$stream->test(Twig_Token::BLOCK_END_TYPE))   
           $values = $this->parser->getExpressionParser()
                          ->parseMultitargetExpression();

        $stream->expect(Twig_Token::BLOCK_END_TYPE);

        $body = $this->parser->subparse(array($this, 'decideMyTagEnd'), true);

        $stream->expect(Twig_Token::BLOCK_END_TYPE);

        return new MyTag_Node($body, $values, $token->getLine(), $this->getTag());
    }

    public function decideMyTagEnd(Twig_Token $token)
    {
        return $token->test('endmytag');
    }

    public function getTag()
    {
        return 'mytag';
    }
}

class MyTag_Node extends Twig_Node
{
    public function __construct(Twig_NodeInterface $body, $values,
                                $line, $tag = null)
    {
        if ($values)
           parent::__construct(array('body' => $body, 'values' => $values),
                               array(), $line, $tag);
        else
           parent::__construct(array('body' => $body), array(), $line, $tag);        
    }

    public function compile(Twig_Compiler $compiler)
    {
        $compiler
            ->addDebugInfo($this)
            ->write("ob_start();\n")
            ->subcompile($this->getNode('body'))            
            ->write("my_func(ob_get_clean()");

        if ($this->hasNode('values'))
        foreach ($this->getNode('values') as $node) {
            $compiler->raw(", ")
                     ->subcompile($node);
        };

        $compiler->raw(");\n"); 
    }
}

function my_func()
{
    $args = func_get_args();
    print_r($args);
}

$loader = new Twig_Loader_String();
$twig = new Twig_Environment($loader);
$twig->addTokenParser(new MyTag_TokenParser()); 

$template =<<<TEMPLATE
{% mytag %}
test1
{% endmytag %}

{% mytag 'var1' %}
test2
{% endmytag %}

TEMPLATE;

echo $twig->render($template);