Julia Approach to python equivalent list of lists

Question

I just started tinkering with Julia and I'm really getting to like it. However, I am running into a road block. For example, in Python (although not very efficient or pythonic), I would create an empty list and append a list of a known size and type, and then convert to a NumPy array:

Python Snippet

a = []
for ....
    a.append([1.,2.,3.,4.])

b = numpy.array(a)

I want to be able to do something similar in Julia, but I can't seem to figure it out. This is what I have so far:

Julia snippet

a = Array{Float64}[]
for .....
    push!(a,[1.,2.,3.,4.])
end

The result is an n-element Array{Array{Float64,N},1} of size (n,), but I would like it to be an nx4 Array{Float64,2}.

Any suggestions or better way of doing this?

Answer 1

The literal translation of your code would be

# Building up as rows
a = [1. 2. 3. 4.]
for i in 1:3
    a = vcat(a, [1. 2. 3. 4.])
end

# Building up as columns
b = [1.,2.,3.,4.]
for i in 1:3
    b = hcat(b, [1.,2.,3.,4.])
end

But this isn't a natural pattern in Julia, you'd do something like

A = zeros(4,4)
for i in 1:4, j in 1:4
    A[i,j] = j
end

or even

A = Float64[j for i in 1:4, j in 1:4]

Basically allocating all the memory at once.

Answer 2

Does this do what you want?

julia> a = Array{Float64}[]
0-element Array{Array{Float64,N},1}

julia> for i=1:3
           push!(a,[1.,2.,3.,4.])
       end

julia> a
3-element Array{Array{Float64,N},1}:
 [1.0,2.0,3.0,4.0]
 [1.0,2.0,3.0,4.0]
 [1.0,2.0,3.0,4.0]

julia> b = hcat(a...)'
3x4 Array{Float64,2}:
 1.0  2.0  3.0  4.0
 1.0  2.0  3.0  4.0
 1.0  2.0  3.0  4.0

It seems to match the python output:

In [9]: a = []

In [10]: for i in range(3):
    a.append([1, 2, 3, 4])
   ....:

In [11]: b = numpy.array(a); b
Out[11]:
array([[1, 2, 3, 4],
       [1, 2, 3, 4],
       [1, 2, 3, 4]])

I should add that this is probably not what you actually want to be doing as the hcat(a...)' can be expensive if a has many elements. Is there a reason not to use a 2d array from the beginning? Perhaps more context to the question (i.e. the code you are actually trying to write) would help.

Answer 3

The other answers don't work if the number of loop iterations isn't known in advance, or assume that the underlying arrays being merged are one-dimensional. It seems Julia lacks a built-in function for "take this list of N-D arrays and return me a new (N+1)-D array".

Julia requires a different concatenation solution depending on the dimension of the underlying data. So, for example, if the underlying elements of a are vectors, one can use hcat(a) or cat(a,dims=2). But, if a is e.g a 2D array, one must use cat(a,dims=3), etc. The dims argument to cat is not optional, and there is no default value to indicate "the last dimension".

Here is a helper function that mimics the np.array functionality for this use case. (I called it collapse instead of array, because it doesn't behave quite the same way as np.array)

function collapse(x)
    return cat(x...,dims=length(size(x[1]))+1)
end

One would use this as

a = []
for ...
    ... compute new_a...
    push!(a,new_a)
end
a = collapse(a)