Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How should I close an hdf5 file after loading in R?

Tags:

r

hdf5

I have successfully installed the hdf5 package on a linux machine. I now wish to read in data from a large number of hdf5 files in a loop and create a time series. Each hdf5 file corresponds to a different time. After reading in many files (just over 1000) R says too many files are open. I would like to find a way of closing them so that the loop can continue. Here is the code:

    fdate <- "200605312355" # the first date for my test loop
    fmax <- 1400
    arr <- vector()

    for (i in 1:fmax){
      fname <- paste(fdate,"_.h5") # path of h5 file
      fid <- hdf5load(fname) # fid = NULL
      arr[i] <- somevariable$data_array[lon,lat]
      # would like to close the file here
      fdate <- newdate(fdate,60*5) # a function that increments the date by seconds.
    }

The hdf5 package contains the function hdf5cleanup, which looks like it might clean things up but it requires a file identifier. The fid in my code above returns NULL. Attempts to insert the file name instead i.e. hdf5cleanup(fname) leads R to abort. Perhaps hdf5load is supposed to close the file and fails. If that is the case, is there any way to close the connection by issuing a system() command or otherwise?

Incidently, showConnections() returns nothing, well, literally just the header names "description class mode text isopen can read can write".

My question in short: How do I close the connection to an hdf5 file in R after loading it with hdf5load()?

like image 781
user1714900 Avatar asked Nov 03 '22 14:11

user1714900


1 Answers

NOTE: According to the comments, the following answer will not work. Leaving it, at least for now, to mark an unsuccessful route to pursue.


I don't have hdf5 installed, so I can't check if this works, so this is a bit of a shot in the dark:

fname <- paste(fdate,"_.h5") # path of h5 file
# fhandle <- open(fname) # Comment pointed out this was not the intended function
fhandle <- file(description = fname, open = "rb")
hdf5load(fhandle) # hdf5load returns nothing when load=TRUE (the default)
arr[i] <- somevariable$data_array[lon,lat]
close(fhandle)

The documentation says that hdf5load takes a filename, but it may also take a filehandle. If so, this might work.

like image 109
Brian Diggs Avatar answered Nov 15 '22 07:11

Brian Diggs