How to automate testing of Tridion templates (with TOM.NET)

Question

I have a recurring problem in templating projects. I can't really test my work in any other way than running the templates in Template Builder. This is a major problem if I'm working on a TBB that is used on several different templates because it means that after changing the code in the TBB I should retest all the templates (and probably with several different pages/components as there might be slightly different cases depending on the content).

As you can see in big projects where TBBs are reused a lot changing them costs a lot of time due to the amount of testing necessary and I would be eager to find a solution for this. I know that unit testing is virtually impossible with the current TOM.NET (most classes/methods are internal) so what could be an alternative way to achieve automated testing?

One solution that I have looked into is to use Core Service to initiate rendering process of a template with some test content and then check if the output is as expected but achieving this requires quite a lot of code and thus produces unwanted overhead (I think it still takes less time than manually retesting the cases). Also this doesn't really allow you to test individual TBBs unless you (programmatically) create separate templates with individual (or a subset of) TBBs. The good thing of this solution is that you could run the tests on your local laptop while developing, assuming you can connect to Tridion-server (you'd still have to upload your code to Tridion before running the tests so its not completely ideal solution).

I know that other alternative is to use DD4T/CWA where you can pretty much handle all the testing in the front-end as the templates are (usually) quite simple.

Any other ideas?

Answer 1

I agree that the emphasis is on automated testing rather than unit testing (which, after all, is mostly about object oriented programming). With Tridion work, it's about transforming data. What you need to test data transforms is to have known inputs, and to be able to make assertions about the outputs. I've tried various approaches over the years, but the most effective so far has been the following:

1) For every template, keep test content in a dedicated Folder, and test pages in a dedicated Structure Group. The content is the input to your tests, and isn't intended to change unless the test requirements change. 2) Put the components on the pages. Publish the pages. Keep it simple: you can often have a page for a single test scenario. You can automate publishing the pages if that helps. 3) Use web testing tools to verify the output. This could be HtmlUnit, Selenium or whatever.

Basically - Tridion is an engine for executing transforms. You don't need a specialised test execution engine for this part, although it's useful to use one for testing the output.

Mocking the package sounds attractive, but as Vesa says, it can turn into a huge amount of work. The simple approach I have outlined works in practice, and was proved on a significant project. You could add variations on the theme if you like: one thing I've considered, but never done on a project, is to use the blueprint to give you more isolation. For example, you could test your page templates by localising your component templates to generate static and predictable component presentations. Suffice it to say that there's enough scope for creativity once you unshackle yourself from the baggage of unit testing approaches.

Answer 2

I have some experience with the CoreService scenario. You will just need to write some helpers to upload your templates, create coumpound templates and run it. The tricky part, however, is verification.

You will need to write some test templates that will help you with verification. One way is to write .Net template that you will pass expected values to and it will do the verification. The other way is to write DreamWeaver template that will print values from package and you will then check it against expected. The advantage of this method is that these values will be returned to you as the result of CoreService Render action and you can do all the verification on the client side.

But the most difficult part is the dataset creation. It will probably take most of your time.

Answer 3

You could try to isolate the majority of the code in classes that can be unit tested.

I guess the main problem here is that Engine and Package are sealed, so you cannot easily mock them up. But you can minimize the interaction with those objects and put the meat of your code in classes that take the relevant input and return the output that should be put in the package etc.

I think you could get a lot of coverage of your TBBs just from unit tests with this approach.

Answer 4

At a customer I've seen an implementation where the tests are invoking the same webservice that Template Builder uses, and they use these to execute the templates, evaluate the results, etc.

Probably worth exploring.

Answer 5

I would suggest writing your own TestRunner with 2 goals: Create test data and run tests.

Create test data: The idea is to create a sample dataset (all fields, some fields, and only mandatory fields automatically). (Bonus points for using Chuck Norris quotes instead of lorem ipsum). The title of the Sample content uses a naming scheme - like [TestContent] and/or is in its' own folder with metadata attached (to find it later).

Create test pages: Find the TestContent. Use GetListUsingItems to find pages where the template is used. Copy the page, and paste it into a TestContent StructureGroup, save. Open the page, add the test content, remove the other content, and save page with special naming schema.

Run tests: Find the TestContent, preview each one, write out report with rendering time, success status, and # of chars.