How to setup testthat for R CMD check?

Question

There are apparently two ways to integrate testthat with R CMD check. I can't get either to work.

Approach #1: (perhaps deprecated)

According to the devtools wiki:

When developing a package, put your tests in inst/tests and then create a file tests/run-all.R (note that it must be a capital R), which contains the following code:

library(testthat) 
library(mypackage)
test_package("mypackage") 

This will evaluate your tests in the package namespace (so you can test non-exported functions), and it will throw an error if there are any test failures. This means you'll see the full report of test failures and R CMD check won't pass unless all tests pass.

The whole package is here. In it are the two files:

## minimalbugexample/inst/tests/run-all.R
library(testthat)
library(minimalbugexample)
test_package('minimalbugexample')

and

## minimalbugexample/inst/tests/test-use-Matrix-package.R
context("Intentional break")
  expect_that( TRUE, equals(FALSE))

my DESCRIPTION is

Package: minimalbugexample
Title: 
Description: 
Version: 0.1.1
Author: Nathan VanHoudnos <[email protected]>
Maintainer: Nathan VanHoudnos <[email protected]>
Depends:
    R (>= 3.0.1),
    Matrix (>= 1.0)
Suggests:
    testthat
License: GPL
LazyData: true
Collate:
    'minimalbugexample-package.r'
    'use-Matrix-package.R'

After installing the package, I can run the tests just fine (they fail, as expected).

> test_package('minimalbugexample')
Intentional break : 1


1. Failure:  -------------------------------------------------------------------
TRUE not equal to FALSE
1 element mismatch
Error: Test failures
> 

But an R CMD check doesn't run the tests.

$ R CMD check minimalbugexample_0.1.1.tar.gz 
... snip ...
* checking PDF version of manual ... WARNING
WARNING: There was 1 warning.
See
  ‘/home/nathanvan/software/minimalbugexample.Rcheck/00check.log’
for details.

I don't think that the PDF warning has anything to do with this, but I can provide more details if requested.

Approach #2: (bleeding edge)

According to the README file of the testthat repository:

Now, recommend practice is to put your tests in tests/testthat, and ensure R CMD check runs then by putting the following code in tests/test-all.R:

library(testthat)
test_check(yourpackage)

So I made sure I had the most recent version of testthat installed:

> install_github("testthat")

And then changed the package. You can get this version here. I modified the two files to be

## minimalbugexample/inst/tests/test-all.R
library(testthat)
test_check(minimalbugexample)

and

## minimalbugexample/inst/tests/testthat/test-use-Matrix-package.R
context("Intentional break")
  expect_that( TRUE, equals(FALSE))

Then updating the package version to 0.1.2 in the DESCRIPTION file, I can build it, install it, and use testthat to check it and get the same output as before. So it seems that as far as testthat is concerned, its working.

However, R CMD check still doesn't run the tests:

$ R CMD check minimalbugexample_0.1.2.tar.gz 
... snip ...
* checking PDF version of manual ... WARNING
LaTeX errors when creating PDF version.
This typically indicates Rd problems.
WARNING: There was 1 warning.
See
  ‘/home/nathanvan/software/minimalbugexample.Rcheck/00check.log’
for details.

So the question:

What am I doing wrong? My preference is for a solution for Approach 2, but I'll take either!

Answer 1

You don't have a tests directory. test-all.R should be located at minimalbugexample/tests/test-all.R.

Then your actual tests go in minimalbugexample/inst/tests for approach #1 or minimalbugexample/tests/testthat/ for approach #2.

For approach #2, the test-all.R file should use test_check(yourpackage) instead of test_package(yourpackage) and the library(yourpackage) call is no longer required.