Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

R reticulate, how do I clear a python object from memory?

I've created a function using some python functionality via the reticulate package, specifically opening an image using PIL:

image <- "~/Desktop/image.jpg"
pil.image <- reticulate::import( "PIL.Image", convert = FALSE )
img <- pil.image$open( image )

I then do a few things with the image (I'm extracting several crops), which works fine. Here's an example of what I'm doing (outputs is a data frame of crops I need, so crop.grid is just a vector of 4 numbers.

crop.grid <- c( outputs$x.start[x],
                outputs$y.start[x],
                outputs$x.stop[x],
                outputs$y.stop[x] )
crop.grid <- as.integer( crop.grid )
crop.grid <- reticulate::r_to_py( crop.grid )
output.array <- img$crop( box = crop.grid )
output.array$save( output.filename )

After this, I want to clear the image from memory (the images I'm opening are very large, so take a lot of memory). I try closing the image in python:

img$close()

As well as in R:

rm( img )
gc()

And replacing the object with something I know to be very small.

img <- reticulate::r_to_py( 1L )

All of these things run fine, but my RAM still registers as being very full. I try them with each of the python objects I've created, but the only thing that clears the RAM effectively is restarting the R session.

I know that within python I'd be better off opening the image using with in order to clear it at the end of the process, but I'm not sure how to implement that using reticulate.


--- UPDATE with an analogous python version:

If I do the above in python directly:

from PIL import Image
img = Image.open( "/home/user/Desktop/image.jpg" )
output = img.crop( [0,0,100,100] )

And then close things:

output.close()
img.close()

The memory clears. The same thing doesn't work from within R though. ie:

output.array$close()
img$close()
gc() # for good measure

Doesn't clear the memory.

like image 512
rosscova Avatar asked Jun 06 '17 00:06

rosscova


Video Answer


1 Answers

You need to do 3 things:

  1. Explicitly create the object in Python:

    py_env <- py_run_string(
        paste(
            "from PIL import Image",
            "img = Image.open('~/Desktop/image.jpg')",
            sep = "\n"
        ),
        convert = FALSE
    )
    img <- py_env$img
    
  2. When you're done with the image, first delete the Python object.

    py_run_string("del img")
    
  3. Then run the Python garbage collector.

    py_gc <- import("gc")
    py_gc$collect()
    

Steps 2 and 3 are the important ones. Step 1 is just so you have a name to delete. If there's a way to delete "implicit" Python objects (can't think of a better term), that would save some boilerplate.

like image 70
Nathan Werth Avatar answered Sep 19 '22 06:09

Nathan Werth