R CMD check does not respect selective code evaluation in knitr code chunks

Question

I am building a package in R 3.1.0 on Windows 7 32-bit (and also on a different machine running Windows 8 64bit) and I am using knitr to write vignettes using Markdown. I am hoping to publish this package on CRAN so I am using R CMD check to check that my demos, datasets and vignettes all behave properly. I keep my vignettes as .Rmd files (rather than making them outside of the package building process and stashing them in inst/doc) because they serve as extra tests on my package and aren't very big anyway.

My problem is that R CMD check fails when building my vignettes, even though if I run R CMD build and then R CMD INSTALL --build everything works out fine. Looking at the log file, it appears to be failing because it tries to evaluate code that I have explicitly told knitr NOT to evaluate. As a generic example, if I write

```{r example-chunk eval=c(1:3, 5:6), tidy=FALSE}
foo = 5
bar = 3
## don't evaluate the next line
file.show("i/dont/really/exist.csv")
## ok, start evaluating again
foobar = foo*bar
```

In a .Rmd file, running R CMD check will fail because it will try to evaluate line 4. However, the chunk will be correctly evaluated if I run R CMD build mypackage and then R CMD install --build mypackage.tar.gz (I know this because I can go to my Rlibs folder and find the flawless html vignettes in mypackage/doc. Similarly, the chunk will also be evaluated correctly I if run R CMD Sweave to build the vignette.

If you want to try this yourself, the package I am building (where I am running into the issue) is on Github: https://github.com/mkoohafkan/flowdurr-edu. You can look at raw/packagemaker.html for instructions, hopefully it's straightforward (the R code runs through the process of making the package directory, building the help files and copying some manually edited files into the package directory). R CMD check fails on all of my vignettes: when building flowdurr-datasets.Rmd, it insists on evaluating a line with a fake path even though I told it not to. When building hspf-analysis.Rmd, R CMD check insists on evaluating a line I excluded because it takes a really long time to complete (using rgenoud to fit some distribution parameters). R CMD check also fails on vignette.Rmd, but for a different reason; I purposely throw errors to show examples of what you can't do with a particular function, and while knitr doesn't have a problem with it R CMD check sure does!

EDIT: My build script was getting some hate so I made this dummy package that reproduced the problem on both of my machines. It illustrates that 1) R CMD check evaluates a line that it shouldn't, and 2) R CMD check does not support error evaluation in a vignette, even though knitr will write error output without issue.

So I guess my question is: What is it that R CMD check is doing differently from R CMD Sweave and R CMD build/install when it comes to vignette building, and is there anything I can do to make R CMD check respect knitr's 'eval' specification? Note that if I use eval=FALSE, R CMD check will respect it and everything is fine; the problem only occurs if I try selective evaluation of a chunk.

Answer 1

I have added vignette engines with the suffix _notangle in the knitr development version 1.6.2. For the original vignette engine knitr::foo, you can use the new engine knitr::foo_notangle to disable tangle (e.g. knitr::knitr_notangle, knitr::rmarkdown_notangle, ...).

If you do not want to wait for the next version of knitr to be on CRAN (which might take a while), you can certainly register a package vignette engine by yourself. Hint: you can make use of existing engines in tools::vignetteEngine(package = 'knitr') so you do not have to completely redefine the knitr vignette engines.

Answer 2

It seems that the issue is more nuanced than I originally thought, so this might not get resolved anytime soon. my workaround is to:

1. manually build the vignettes using R CMD Sweave
2. Copy the HTML outputs to inst/doc
3. Delete the vignettes folder (or add entries to .Rbuildignore--thanks @Ben!)
4. Build and check

It's not ideal, but right now it looks like the only way for my package to get through CRAN checks.