Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Confused by memory allocation and garbage collection in Julia

I am a bit confused by memory allocation in Julia. I know from the FAQ that clearing the memory used by a large variable is done by setting it to something small (like 0) and then running gc().

However, I'm a bit confused by the following. I create a random Float32 array:

@time A = rand(Float32, 10000, 10000);

time indicates that ~400MB of RAM was allocated, and Julia's RAM usage increases by 400MB. This makes sense.

I then apply fft, but don't bind the result to any variable:

@time fft(A);

time indicates that ~800MB of RAM was allocated, and Julia's RAM usage increases by 800MB.

However, the RAM usage remains at 1.2GB higher than at the start. And that confuses me, because I didn't equate any variable to fft(A), so I would expect that the 800MB allocated would be immediately freed after the fft was executed.

I tried to run gc, thinking that Julia would realize that there was an additional 800MB of RAM that was being used for nothing:

gc();

This does nothing. RAM usage remains at ~1.3GB.

However, the following two lines,

A = 0;
gc();

frees all 1.2GB that is in use, despite the fact that sizeof(A) is only 400MB. So my question is:

  • Why does it appear as though an object which is 400MB in size (according to sizeof) actually has 1.2GB allocated to it?
like image 647
DumpsterDoofus Avatar asked Sep 08 '14 15:09

DumpsterDoofus


People also ask

Does Julia use garbage collection?

Julia's garbage collector algorithm is called mark and sweep. This algorithm consists of two phases: the mark phase, where all objects that are not garbage are found and marked so; and the sweep phase, where all unmarked objects are cleaned.

What is memory allocation and garbage collection in data structure?

Memory allocation is the act of asking for some memory to the system to use it for something. Garbage collection is a process to check if some memory that was previously allocated is no longer really in use (i.e. is no longer accessible from the program) to free it automatically.

How does the garbage collector determine which objects to remove from memory?

When the garbage collector performs a collection, it releases the memory for objects that are no longer being used by the application. It determines which objects are no longer being used by examining the application's roots.


1 Answers

Each command returns something, even if it is only nothing. ans is assigned to each returned object even if there is no direct assignment and even if the command ends with a semicolon.

** EDIT ** [Updated info for Julia version ≥ v0.7.0]

Use varinfo() for Julia v0.7.0 and higher (whos() for Julia v0.6.4 and lower) between commands to watch assignments and the allocated space.

like image 114
rickhg12hs Avatar answered Sep 24 '22 22:09

rickhg12hs