How can I shuffle a range of numbers and then split it into subarrays of a certain length?

Question

Suppose I have a range 1:N, I want to shuffle the range into a random order and then split the resulting shuffled array into subarrays that are at most 128 elements long. How can I do this?

This question is based by one that appeared on the JuliaLang slack channel.

Answer 1

The function shuffle from the Random standard library can be used to shuffle a container into random order:

julia> using Random: shuffle

julia> shuffle(1:10)
10-element Array{Int64,1}:
  6
  9
  3
  2
 10
  1
  8
  7
  5
  4

The function Iterators.partition from Julia's Base can be used to iterate over an iterable in chunks of a fixed length:

julia> using Base.Iterators: partition

julia> partition(1:20, 7)
Base.Iterators.PartitionIterator{UnitRange{Int64}}(1:20, 7)

However, partition returns a lazy iterator by default so if we want to materialize the actual result, we'll need to collect it:

julia> collect(partition(1:20, 7))
3-element Array{UnitRange{Int64},1}:
 1:7
 8:14
 15:20

Putting this all together, we have

julia> using Random: shuffle

julia> using Base.Iterators: partition

julia> shuffle_partition(N; chunk_size=128) = (collect ∘ partition)(shuffle(1:N), chunk_size)
shuffle_partition (generic function with 1 method)

julia> shuffle_partition(503)
4-element Array{SubArray{Int64,1,Array{Int64,1},Tuple{UnitRange{Int64}},true},1}:
 [313, 51, 117, 373, 381, 340, 342, 415, 423, 453  …  201, 178, 167, 242, 2, 76, 146, 439, 363, 448]
 [115, 121, 306, 440, 295, 181, 30, 280, 388, 227  …  362, 39, 317, 171, 55, 214, 261, 251, 96, 9]
 [486, 248, 161, 319, 325, 176, 80, 369, 434, 209  …  442, 350, 273, 419, 130, 305, 192, 482, 265, 234]
 [460, 31, 400, 466, 220, 447, 119, 446, 198, 141  …  226, 438, 74, 152, 203, 303, 378, 231, 458, 194]

julia> length.(ans)
4-element Array{Int64,1}:
 128
 128
 128
 119

---

This answer is based on the answer found on Slack.

Answer 2

using Iterators: I think the most simple is using randperm (if the values are between 1 and N), so

using Base.Iterators: partition
using Random: randperm

N = 513
k = 128

collect(partition(randperm(N), k))

should work.

Answer 3

parts = view.(Ref(shuffle(1:N)),(i:min(i+k-1, N) for i in 1:k:N))

This assumes N is the number of elements and parition size is k. The obtained result is a list of views (hence shuffled 1:N is stored only once in the memory). Note how Ref is used to avoid vectorization over the shuffled list.

Sample test code:

julia> using Random

julia> N, k = 20, 4;

julia> parts = view.(Ref(shuffle(1:N)),(i:min(i+k-1, N) for i in 1:k:N))
5-element Array{SubArray{Int64,1,Array{Int64,1},Tuple{UnitRange{Int64}},true},1}:
 [18, 15, 1, 6]
 [10, 20, 4, 14]
 [17, 9, 19, 16]
 [5, 8, 12, 3]
 [11, 13, 2, 7]