Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

An array of dictionaries into a DataFrame at ONE go in Julia

Tags:

julia

I have an array of dictionaries as follows:

julia> data  
2-element Array{Any,1}:  
 Dict{String,Any}("ratio1"=>1.36233,"time"=>"2014-06-19T15:47:40.000000Z","ratio2"=>1.36243)
 Dict{String,Any}("ratio1"=>1.3623,"time"=>"2014-06-19T15:48:00.000000Z","ratio2"=>1.36245)

How can I pop this into a DataFrame at ONE go without looping through each dictionary and key one by one, so that I have a Dataframe like below:

2×3 DataFrame
│ Row │ ratio1  │ ratio2  │ time                        │
│     │ Float64 │ Float64 │ String                      │
├─────┼─────────┼─────────┼─────────────────────────────┤
│ 1   │ 1.36233 │ 1.36243 │ 2014-06-19T15:47:40.000000Z │
│ 2   │ 1.3623  │ 1.36245 │ 2014-06-19T15:48:00.000000Z │
like image 204
Julia O Avatar asked Jan 13 '19 11:01

Julia O


1 Answers

One way would be

julia> vcat(DataFrame.(data)...)
2×3 DataFrame
│ Row │ ratio1  │ ratio2  │ time                        │
│     │ Float64 │ Float64 │ String                      │
├─────┼─────────┼─────────┼─────────────────────────────┤
│ 1   │ 1.36233 │ 1.36243 │ 2014-06-19T15:47:40.000000Z │
│ 2   │ 1.3623  │ 1.36245 │ 2014-06-19T15:48:00.000000Z │

julia> @btime vcat(DataFrame.($data)...)
  31.146 μs (157 allocations: 12.19 KiB)

i.e. convert every Dict to a DataFrame and concatenate them.

More detailed explanation:

  • DataFrame(somedict) is a constructor call which creates a DataFrame from a Dict
  • DataFrame.(arrayofdicts): The dot here broadcasts the constructor call such that all contained Dicts get converted to DataFrames and we obtain an array of DataFrames. See Dot Syntax for Vectorizing Functions in the Julia docs for more information.

  • vcat(DataFrame.(arrayofdicts)...): We now splat our array of DataFrames into the vcat function, which concatenates them vertically (rows).

like image 172
carstenbauer Avatar answered Nov 20 '22 11:11

carstenbauer