I'm processing a massive TSV (tab separated values) file and want to do this as efficiently as possible. To that end, I thought I'd prevent allocation of a new Vec
for every line by pre-allocating it before the loop:
let mut line = String::new();
let mut fields = Vec::with_capacity(headers.len());
while reader.read_line(&mut line)? > 0 {
fields.extend(line.split('\t'));
// do something with fields
fields.clear();
}
Naturally, the borrow checker isn't amused, because we're overwriting line
while fields
may still have references into it:
error[E0502]: cannot borrow `line` as mutable because it is also borrowed as immutable
--> src/main.rs:66:28
|
66 | while reader.read_line(&mut line)? > 0 {
| ^^^^^^^^^ mutable borrow occurs here
67 | fields.extend(line.split('\t'));
| ------ ---- immutable borrow occurs here
| |
| immutable borrow later used here
(Playground)
This isn't actually a problem because fields.clear();
removes all references, so at the start of the loop when read_line(&mut line)
is called, fields
does not actually borrow anything from line
.
But how do I inform the borrow checker of this?
Your problem looks similar to the one described in this post.
In addition to the answers there (lifetime transmutations, refcells), depending on the Complex Operation you commented out, you might not need to store references to line
at all. Consider, for example, the following modification of your playground code:
use std::io::BufRead;
fn main() -> Result<(), std::io::Error> {
let headers = vec![1,2,3,4];
let mut reader = std::io::BufReader::new(std::fs::File::open("foo.txt")?);
let mut fields = Vec::with_capacity(headers.len());
loop {
let mut line = String::new();
if reader.read_line(&mut line)? == 0 {
break;
}
fields.push(0);
fields.extend(line.match_indices('\t').map(|x| x.0 + 1));
// do something with fields
// each element of fields starts a field; you can use the next
// element of fields to find the end of the field.
// (make sure to account for the \t, and the last field having no
// 'next' element in fields.
fields.clear();
}
Ok(())
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With