How to produce jUnit fie with R testthat

Question

I have R code (not a package) which I would like to cover with acceptance tests using testthat with output being used in Jenkins.

I can start with two files which demonstrate the code structure:

# -- test.R
source("test-mulitplication.R")

# -- test-mulitplication.R
library(testthat)
test_that("Multipilation works ", {
  res <- 5 * 2
  expect_equal(res, 10)
})

After the run I would like to get an xml file with results per test file or all tests in a single file.

I noticed there is a reporter capability within testthat, but most of it seems to be internal to the package. It is not clear how to save test results and how flexible is the functionality.

Unfortunately documentation on that part is not complete.

EDIT

I have now found a way to test the directory with better syntax and option for junit output:

# -- tests/accpetance-tests.R
options(testthat.junit.output_file = "test-out.xml")
test_dir("tests/")

# -- tests/test-mulitplication.R
library(testthat)
test_that("Multipilation works ", {
  res <- 5 * 2
  expect_equal(res, 10)
})

This I believe produces an XML object inside the reporter, but I still don't see how to save it to a file.

I tried to wrap the test_dir call with with_reporter, but that does not do much.

Answer 1

Update 2019:

Version 2.1.0 of testthat no longer needs context in order to work correctly. So, I would expect the question from your original code to work correctly.

Source: https://www.tidyverse.org/articles/2019/04/testthat-2-1-0/

Original Answer:

There was testthat commit 4 days ago which references this functionality. A new option is being introduced in the development version of testthat.

If you run:

devtools::install_github("r-lib/testthat")
options(testthat.output_file = "test-out.xml")
test_dir("tests/")

this should produce an file in your working directory.

The catch is it may not work with the reporter you want. With the devtools version of testthat installed:

options(testthat.output_file = "test-out.xml")
test_dir("tests/", reporter = "junit")

produces an error regarding xml2. Trying the dev branch of xml2 did not resolve the issue. Given that this change is fairly recent, it might be worth filing an issue over on github.

Not sure if this gets you any closer, but we're getting a report to output which is a start!

EDIT

This works, but you need to be sure and add a "context" to the top of your test or else you'll get an error. Try changing the top of your multiplication test to something like:

# -- test-mulitplication.R
library(testthat)
context("Testing Succeeded!")
test_that("Multipilation works ", {
  res <- 5 * 2
  expect_equal(res, 10)
})

context("Test Failed!")
test_that("Multipilation works ", {
  res <- 5 * 2
  expect_equal(res, 12)
})

and then re-run:

options(testthat.output_file = "test-out.xml")
test_dir("tests/", reporter = "junit")

that worked for me! For some reason not including a context header causes problems. This is probably by design though.

Answer 2

The solution with test_dir("tests/", reporter = "junit") prints out in result testthat.Rout. We could use sink() to write it to other file, but it is a workaround.

A better way is to call JunitReporter object directly and specify parameter where to put report:

library(testthat)
library(packageToTest)

test_check("packageToTest", reporter = JunitReporter$new(file = "junit_result.xml"))