Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Check size in bytes of variable using Julia

Tags:

julia

Question: How do I check the size in bytes of a variable using Julia?

What I've tried: In Matlab, the whos() function provided this information, but in Julia that just provides the variable names and module. Browsing the standard library in the Julia manual, sizeof() looked promising, but it only appears to provide the size of the canonical binary representation, rather than the current variable.

like image 631
Colin T Bowers Avatar asked Feb 09 '15 04:02

Colin T Bowers


3 Answers

sizeof works on variables too

sizeof(a::Array{T,N})

returns the size of the array times the element size.

julia> x = [1 2 3 4]
1x4 Array{Int64,2}:
 1  2  3  4

julia> sizeof(x)
32

julia> x = Int8[1 2 3 4]
1x4 Array{Int8,2}:
 1  2  3  4

julia> sizeof(x)
4

sizeof(B::BitArray{N})

returns chunks; each chunk is 8 bytes so can represent up to 64 bits

julia> x = BitArray(36);
julia> sizeof(x)
8 

julia> x = BitArray(65);
julia> sizeof(x)
16

sizeof(s::ASCIIString) and sizeof(s::UTF8String)

return the number of characters in the string (1 byte/char).

julia> sizeof("hello world")
11

sizeof(s::UTF16String) and sizeof(s::UTF32String)

Same as above but with 2 and 4 bytes/character respectively.

julia> x = utf32("abcd");
julia> sizeof(x)
16

Accordingly other strings

sizeof(s::SubString{ASCIIString}) at string.jl:590
sizeof(s::SubString{UTF8String}) at string.jl:591
sizeof(s::RepString) at string.jl:690
sizeof(s::RevString{T<:AbstractString}) at string.jl:737
sizeof(s::RopeString) at string.jl:802
sizeof(s::AbstractString) at string.jl:71

core values

returns the number of bytes each variable uses

julia> x = Int64(0);
julia> sizeof(x)
8

julia> x = Int8(0);
julia> sizeof(x)
1

julia> x = Float16(0);
julia> sizeof(x)
2

julia> x = sizeof(Float64)
8

one would expect, but note that Julia characters are wide characters

julia> sizeof('a')
4

getBytes

For cases where the layout is more complex and/or not contiguous. Here's a function that will iterate over the fields of a variable (if any) and return of sum of all of the sizeof results which should be the total number of bytes allocated.

getBytes(x::DataType) = sizeof(x);

function getBytes(x)
   total = 0;
   fieldNames = fieldnames(typeof(x));
   if fieldNames == []
      return sizeof(x);
   else
     for fieldName in fieldNames
        total += getBytes(getfield(x,fieldName));
     end
     return total;
   end
end

using it

create an instance of a random-ish type...

julia> type X a::Vector{Int64}; b::Date end

julia> x = X([i for i = 1:50],now())
X([1,2,3,4,5,6,7,8,9,10  …  41,42,43,44,45,46,47,48,49,50],2015-02-09)

julia> getBytes(x)
408
like image 173
waTeim Avatar answered Nov 02 '22 04:11

waTeim


The function Base.summarysize provides exactly that

It also includes the overhead from the struct as seen in the examples.

julia> struct Foo a; b end

julia> Base.summarysize(ones(10000))
80040

julia> Base.summarysize(Foo(ones(10000), 1))
80064

julia> Base.summarysize(Foo(ones(10000), Foo(ones(10, 10), 1)))
80920

However, care should be taken as the function is non-exported and might not be future proof

like image 22
Korbinian Eckstein Avatar answered Nov 02 '22 03:11

Korbinian Eckstein


In julia 1.6, varinfo() shows sizes:

julia> a = 1;
julia> v = ones(10000);
julia> varinfo()
  name                    size summary                      
  –––––––––––––––– ––––––––––– –––––––––––––––––––––––––––––
  Base                         Module                       
  Core                         Module                       
  InteractiveUtils 250.022 KiB Module                       
  Main                         Module                       
  ans               78.164 KiB 10000-element Vector{Float64}
  v                 78.164 KiB 10000-element Vector{Float64}
  a                    8 bytes Int64

For specific variables, either use pattern matching (r"..." is a regular expression):

julia> varinfo(r"^v$")
  name       size summary                      
  –––– –––––––––– –––––––––––––––––––––––––––––
  v    78.164 KiB 10000-element Vector{Float64}

or combine the Base.summarysize from Korbinian answer with Base.format_bytes:

julia> pretty_summarysize(x) = Base.format_bytes(Base.summarysize(x))
pretty_summarysize (generic function with 1 method)

julia> pretty_summarysize(v)
"78.164 KiB"

Edit: beware that summarysize had a bug, at least in 1.5.3 and 1.6.1. varinfo was affected as well. It is fixed (tested with 1.7.3).

like image 2
ederag Avatar answered Nov 02 '22 04:11

ederag