How can I disable scientific notation in fwrite data.table in R?

Question

I would like to disable scientific notation when writing numbers to csv files in data.table. I can do that with write.csv using options(scipen = 999) but not with fwrite. Example:

require(data.table) dt <- data.table("ID" = c("A", "B", "C", "D"), VALUE = c(0.0000001, 0.1234567, 1000000, 1234567)) options(scipen = 999) write.csv(dt, row.names = FALSE) # "ID","VALUE" # "A",0.0000001 # "B",0.1234567 # "C",1000000 # "D",1234567 fwrite(dt, row.names = FALSE) # ID,VALUE # A,1e-07 # B,0.1234567 # C,1e+06 # D,1234567 

I would like line A and C in fwrite to be written as in write.csv. I am using data.table version 1.10.0. Do you know how to do that?

Answer 1

This is now fixed in data.table v1.12.4. Function fwrite gains option scipen. Quote from data.table github site: https://github.com/Rdatatable/data.table/blob/master/NEWS.md

"Gains scipen #2020, the number 1 most-requested feature #3189. The default is getOption("scipen") so that fwrite will now respect R's option in the same way as base::write.csv and base::format, as expected. The parameter and option name have been kept the same as base R's scipen for consistency and to aid online search. It stands for 'scientific penalty'; i.e., the number of characters to add to the width within which non-scientific number format is used if it will fit. A high penalty essentially turns off scientific format. We believe that common practice is to use a value of 999, however, if you do use 999, because your data might include very long numbers such as 10^300, fwrite needs to account for the worst case field width in its buffer allocation per thread. This may impact space or time. If you experience slowdowns or unacceptable memory usage, please pass verbose=TRUE to fwrite, inspect the output, and report the issue. A workaround, until we can determine the best strategy, may be to pass a smaller value to scipen, such as 50. We have observed that fwrite(DT, scipen=50) appears to write 10^50 accurately, unlike base R. However, this may be a happy accident and not apply generally. Further work may be needed in this area."

Answer 2

You can try this:

require(data.table) #Sample data dt <- data.table("ID" = c("A", "B", "C", "D"), VALUE = c(0.0000001, 0.1234567, 1000000, 1234567)) dt     ID        VALUE 1:  A 1.000000e-07 2:  B 1.234567e-01 3:  C 1.000000e+06 4:  D 1.234567e+06 

Idea for a solution:

dt$VALUE <- format(dt$VALUE, scientific = FALSE)  fwrite(dt, row.names = FALSE) 

Results

ID,VALUE A,      0.0000001 B,      0.1234567 C,1000000.0000000 D,1234567.0000000 

Edit - remove the trailing zeros.

In addition, if you'd like to remove the trailing zeros you can add the argument drop0trailing = TRUE to format.

dt$VALUE <- format(dt$VALUE, drop0trailing = TRUE, scientific = FALSE)  fwrite(dt, row.names = FALSE)  ID,VALUE A,      0.0000001 B,      0.1234567 C,1000000 D,1234567