Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Strange output from fread when called from knitr

I'm using the recently introduced fread function from data.table to read data files. When I wrap my code into a knitr (Rmd) document, I noticed some strange output, namely lines like:

## 
0%

even though the verbose option of fread was set to FALSE. I've used sink to hide this output, but I'd like to report the exact problem to the package author(s). Here's a minimal example,

library(knitr)

test = "```{r}
require(data.table) 
fread('1 2 3\n')
```"
knit2html(text=test, output="test.html")
browseURL("test.html")

What is the 0% output?

like image 694
baptiste Avatar asked Mar 12 '13 20:03

baptiste


People also ask

What is the functionality of knitr package?

It is a package in the programming language R that enables integration of R code into LaTeX, LyX, HTML, Markdown, AsciiDoc, and reStructuredText documents. The purpose of knitr is to allow reproducible research in R through the means of literate programming. It is licensed under the GNU General Public License.

How can you compile the R Markdown document using knitr package?

The usual way to compile an R Markdown document is to click the Knit button as shown in Figure 2.1, and the corresponding keyboard shortcut is Ctrl + Shift + K ( Cmd + Shift + K on macOS). Under the hood, RStudio calls the function rmarkdown::render() to render the document in a new R session.

What is knitr in RMarkdown?

RMarkdown is an extension to markdown which includes the ability to embed code chunks and several other extensions useful for writing technical reports. The rmarkdown package extends the knitr package to, in one step, allow conversion between an RMarkdown file (.Rmd) into PDF, HTML, word document, amongst others.


1 Answers

It's a % progress counter. For me it prints 0%, 5%, 10%, ... 95%, 100% (for example) with a \r at the end to make it appear on one line just underneath the call to fread when typed at the prompt.

But when called from functions, batches and knitr this is undesirable. This has now been removed. From NEWS for v1.8.9 (rev 851) :

  • % progress console meter has been removed. The ouput was inconvenient in batch mode, log files and reports which don't handle \r. It was too difficult to detect where fread is being called from, plus, removing it speeds up fread a little by saving code inside the C for loop (which is why it wasn't made optional instead). Use your operating system's system monitor to confirm fread is progressing. Thanks to Baptiste for highlighting :
    Strange output from fread when called from knitr

Just a quick reminder for completeness. From the top of ?fread :

This function is still under development. For example, dates are read as character (they can be converted afterwards using the excellent fasttime package or standard base functions) and embedded quotes ("\"" and """") have problems. There are other known issues that haven't been fixed and features not yet implemented. But, you may find it works in many cases. Please report problems to datatable-help or Stack Overflow's data.table tag.

Not for production use yet. Not because it's unstable in the sense that it crashes or is buggy (your testing will show whether it is stable in your cases or not) but because fread's arguments and behaviour is likely to change in future; i.e., we expect to make (hopefully minor) non-backwards-compatible changes. Why has it been released to CRAN then? Because a maintenance release was asked for by CRAN maintainers to comply with new stricter tests in R-devel, and a few Bioconductor packages depend on data.table and Bioconductor requires packages to pass R-devel checks. It was quicker to leave fread in and write these paragraphs, than take fread out.

like image 112
Matt Dowle Avatar answered Oct 19 '22 07:10

Matt Dowle