Efficiently chunk large vector into a vector of vectors

Question

I want to chunk a large vector into a vector of vectors. I know about chunks(), but am not sure of the best way to go from the iterator to a 2D Vec. I have found the following to work, but is there a better way to write this?

let v: Vec<i32> = vec![1, 1, 1, 2, 2, 2, 3, 3, 3];
let v_chunked: Vec<Vec<i32>> = v.chunks(3).map(|x| x.to_vec()).collect();

println!("{:?}", v_chunked); // [[1, 1, 1], [2, 2, 2], [3, 3, 3]]

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=5031d4d0e43470242b8304d483967a25

An operation similar to this is one of the slowest parts of my program after profiling and I was wondering how to improve it.

Answer 1

If a Vec<Vec<i32>> is what you really want then this is a pretty good way of doing it. Any other approach (excluding unsafe code, see below) is unlikely to be significantly faster or use noticeably less memory. Regardless of the actual code, each nested Vec is a new memory allocation and all the data will be need to copied - and that's essentially all that your code does.

A more "Rusty" way to represent a 2D structure like this is a Vec of slices into the original data. That way you don't do any copying and no new allocations.

let v_slices: Vec<&[i32]> = v.chunks(3).collect();

println!("{:?}", v_slices); // [[1, 1, 1], [2, 2, 2], [3, 3, 3]]

---

_{Edit: I did have an extra bit here with some unsafe code that would transform a Vec<i32> into a Vec<Vec<i32>> without reallocating. However, it has been pointed out that it still has Undefined Behaviour, and that the problem is fundamentally not fixable}