Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Piping Rscript gives error after output

Tags:

unix

r

pipe

I wrote a small R script to read JSON, which works fine but upon piping with

Rscript myscript.R | head

the (full, expected) output comes back with an error

Error: ignoring SIGPIPE signal
Execution halted

Oddly I can't remove this by piping STDERR to /dev/null using:

Rscript myscript.R | head 2>/dev/null

The same error is given... presumably because the error arises within the Rscript command? The suggestion to me is that the output of the head command is all STDOUT.

  • Piping STDOUT to /dev/null returns only the error message
  • Piping STDERR to /dev/null returns only the error message...!

Piping the output to cat seems to be 'invisible' - this doesn't cause an error.

Rscript myscript.R | cat | head

Further pipe chaining is possible after the cat command but it feels like I may be ignoring something important by not addressing the error.

Is there a setting I need to use within the script to permit piping without the error? I'd like to have R scripts at the ready for small tasks as is done with the likes of Python and Perl, and it'd get annoying to always have to add a useless cat.

There is discussion of handling this error in C here, but it's not immediately clear to me how this would relate to an R script.

Edit In response to @lll's answer, the full script in use (above called as 'myscript.R') is

library(RJSONIO)
note.list <- c('abcdefg.json','hijklmn.json')
# unique IDs for markdown notes stored in JSON by Laverna, http://laverna.cc
for (laverna.note in note.list) {
  # note.file <- path.expand(file.path('~/Dropbox/Apps/Laverna/notes',
  #                                   laverna.note))
  # For the purpose of this example run the script in the same
  # directory as the JSON files
  note.file <- path.expand(file.path(getwd(),laverna.note))
  file.conn <- file(note.file)
  suppressWarnings( # warnings re: no terminating newline
    cat(paste0(substr(readLines(file.conn), 2, 15)),'\n') # add said newline
  )
  close(file.conn)
}

Rscript myscript.R outputs

"id":"abcdefg"
"id":"hijklmn" 

Rscript myscript.R | head -1 outputs

"id":"abcdefg" 
Error: ignoring SIGPIPE signal
Execution halted

It's not clear to me what would be terminating 'early' here

Edit 2 It's replicable with readLines so I've removed JSON library-specific details in the example above. Script and dummy JSON gisted here.

Edit 3 It seems it may be possible to take command-line arguments including pipes and pass them to pipe() - I'll try this when I can and resolve the question.

like image 828
Louis Maddox Avatar asked Mar 07 '15 14:03

Louis Maddox


1 Answers

I encountered the same annoying error. It appears to be generated from within R by a function writing to STDOUT, if the R function is still running (outputting data to the pipe) when the pipe stops 'listening'.

So, errors can be suppressed by simply wrapping the R output function into try(...,silent=TRUE), or specifically this error can be handled by wrapping the R output function into a more-involved tryCatch(...,error=...) function.

Example: Here's a script that generates an error when piped:

#! /Library/Frameworks/R.framework/Resources/bin/rscript
random_matrix=matrix(rnorm(2000),1000)
write.table(x=random_matrix,file="",sep=",",row.names=FALSE,col.names=FALSE)

Output when called from bash and piped to head:

./myScript.r | head -n 1
-1.69669866833626,-0.463199773124574
Error in write.table(x = random_matrix, file = "", sep = ",", row.names = FALSE,  :
  ignoring SIGPIPE signal
Execution halted

So: wrap the write.table output function into try to suppress all errors that occur during output:

try(write.table(x=random_matrix,file="",sep=",",row.names=FALSE,col.names=FALSE),silent=TRUE)

Or, more-specific, just suppress the "ignoring SIGPIPE signal" error:

tryCatch(write.table(x=random_matrix,file="",sep=",",row.names=FALSE,col.names=FALSE),
    error=function(e) if(!grepl("ignoring SIGPIPE signal",e$message))stop(e) )
like image 132
Dominic van Essen Avatar answered Oct 10 '22 05:10

Dominic van Essen