Difference between tests/ and inst/tests/ for R packages

Question

I'm developing a new package and I have some unit tests to write. What's the difference between tests/ and inst/tests/? What kinds of stuff should go in each?

In particular, I see in http://journal.r-project.org/archive/2011-1/RJournal_2011-1_Wickham.pdf that Hadley recommends using inst/tests/ "so users also have access to them," then putting a reference in tests/ to run them all. But why not just put them all in tests/?

Answer 1

What @hadley means is that in binary packages, tests/ is not present; it is only in the source package. The convention is that anything in inst/ is copied to the package top-level directory upon installation, so inst/tests/ will be available as /tests in the binary and installed package directory structure.

See my permute package as an example. I used @hadley's testthat package as a learning experience and for my package tests. The package is on CRAN. Grab the source tarball and notice that it has both tests/ and inst/tests/, then grab the Windows binary and notice that only has tests/ and that is a copy of the one from inst/tests in the sources.

Strictly, only tests/ is run by R CMD check etc so during development and as checks in production packages you need code in tests/ to test the package does what it claims, or other unit tests. You can of course have code in tests/ that runs R scripts in /inst/tests/ which actually do the tests, and this has the side effect of also making the test code available to package users.

The way I see things is you need at least tests/, whether you need inst/tests will depend upon how you want to develop your package and what unit testing code/packages you are using. The inst/tests/ is something @hadley advocates but it is far from being a standard across much of CRAN.

Answer 2

As 'Writing R Extensions' says, only inst/tests gets installed. So only those can be used for tests for both the source versions of the package, and its binary (installed) form.

Otherwise tests/ is of course there for the usual R CMD check tests. Now, Martin Maechler once developed a 'hook' script from tests/ to use inst/tests, and I have been using that in a few of my packages, permitting them to be invoked when a user looks at the source, as well as after the fact. So you can in fact pivot out of the one set of tests into the other, and get the best of both worlds.

Edit: Here is a link to what our RProtoBuf package does to invoke inst/tests/ tests from tests/: https://github.com/eddelbuettel/rprotobuf/blob/master/tests/runUnitTests.R And the basic idea is due to Martin as I said.