Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Machine specific slow read_excel read times

Tags:

r

readxl

I'm getting really slow read times using read_excel on a particular machine with a particular xlsm file. Reading the same xlsm file across 2 different machines I get read times of 72 seconds versus 0.2 seconds. Where the slower machine actually has more computational power.

Both machines are Windows and I'm using readxl package version 1.1.0 on both machines. I've tried using package v1.2.0 on the slower machine and the same problem persists.

I had been reading this xlsm file into R for the last 2 years+ without any problems. I had been regularly running a set of macros on the excel file over those 2 years. Now, however, when I run those same macros and save the file, the slower machine now take 70s instead of <1s to read it into R. If I ask the slower machine to read older versions of the xlsm file it has no problems. The file is approximately 15MB. It doesn't matter which tab of the xlsm file I ask read_excel to read, the time is still about 70 seconds.

I've updated R (v3.5.2) and ensured all the readxl package dependencies are up to date too. I'm at a loss as to what else to try, any ideas would be much appreciated.

like image 701
Will T-E Avatar asked Nov 06 '22 22:11

Will T-E


1 Answers

Edit: When I tested my below solution it worked once and then never again after that.

New solution: The only real solution I can find is reverting to an older version of readxl.

1) Install RTools 3.5 - https://cran.r-project.org/bin/windows/Rtools/

2) Install devtools package - install.packages("devtools")

3) Install readxl version 1.2.0 - devtools::install_version("readxl", "1.2.0")

Incorrect solution:

A similar thing has been happening to me and I recently managed to "fix" it by deleting all of the workbook/worksheet 'names' in the excel file before trying to load it in with readxl.

Some background: People had added a bunch of junk names referencing external connections and I'm wondering if readxl was trying to resolve these before loading in the data causing it to hang. I don't know why it only hangs on my current computer while the same file won't hang when loading on another computer.

like image 194
Anthony Bucci Avatar answered Nov 15 '22 07:11

Anthony Bucci