I have the next code:
struct Tokenizer {
reader: BufReader<File>,
buf: Vec<u8>,
token: String
}
impl Tokenizer {
fn new(path: &PathBuf) -> Tokenizer {
let file = File::open(path).expect("Unable to open file!");
Tokenizer {
reader: BufReader::new(file),
buf: Vec::<u8>::new(),
token: String::new()
}
}
fn next(&mut self) -> bool {
if self.buf.len() == 0 {
if self.reader.read_until(b'\n', &mut self.buf)
.expect("Unable to read file") == 0 {
return false;
}
}
let s = String::from_utf8(self.buf).expect("Unable to read file");
let mut start: i8 = -1;
let mut end: i8 = -1;
for (i, c) in s.char_indices() {
if start == -1 {
if !c.is_whitespace() {
start = i as i8;
}
} else {
if c.is_whitespace() {
end = i as i8;
}
}
}
self.token = s.chars().skip(start as usize).take((end - start) as usize).collect();
self.buf = s.into_bytes();
self.buf.clear();
return true;
}
}
But it doesn't work due to error:
error[E0507]: cannot move out of `self.buf` which is behind a mutable reference
--> src\parser.rs:28:35
|
28 | let s = String::from_utf8(self.buf).expect("Unable to read file");
| ^^^^^^^^ move occurs because `self.buf` has type `std::vec::Vec<u8>`, which does not implement the `Copy` trait
I've read similar questions that advice to use swap
but I didn't get how it will help me. How to fix this error and why does it not compile?
String::from_utf8
takes its parameter by value, meaning it does not merely use the vec it's given it consumes it.
But here it can't consume the input: since self
is just a mutable borrow self.buf
is at most a mutable borrow (aka &mut Vec<u8>
), and thus the signatures don't match. This is because String::from_utf8
will check if the input is valid and then just reinterpret it as a valid string, avoiding new allocations.
There are two easy ways to fix this issue I can see here:
The simplest is to use a function which takes takes a reference as input like str::from_utf8
, this will allocate a new String internally in order to return the data, therefore is a bit less efficient but more convenient.
The alternative -- and the swap
suggestion you got -- is to move the data out of the borrow: if you have a mutable borrow you can't consume self.buf
but you can replace it with a new (owned) buffer.
swap
performs this operation and gives back the old value, so you can swap the current content of self.buf
for a brand new (empty) vector, and get out the vector formerly known as self.buf
, now owned by you (instead of self
) and thus consumable.
The way it would look here is:
let mut buf = Vec::new();
swap(&mut buf, self.buf);
// here we now consume the swapped buf
let s = String::from_utf8(buf).expect("Unable to read file");
let mut start: i8 = -1;
let mut end: i8 = -1;
for (i, c) in s.char_indices() {
if start == -1 {
if !c.is_whitespace() {
start = i as i8;
}
} else {
if c.is_whitespace() {
end = i as i8;
}
}
}
self.token = s.chars().skip(start as usize).take((end - start) as usize).collect();
self.buf = s.into_bytes();
self.buf.clear();
return true;
Addendum: since 1.40, there's also a mem::take
utility function which will simply swap the value with its default (so only works for Default
types).
Since Vec<u8>
is Default
, you could replace
let mut buf = Vec::new();
swap(&mut buf, self.buf);
with
let buf = take(self.buf);
You're getting this error as String::from_utf8
tries to take ownership of it's parameter. In your case it's property of &mut self
, which is referred (you're planning to use this struct in future, not consume it).
Easy solution would be to use other function which takes reference:
fn next(&mut self) -> bool {
...
let s = String::from_utf8_lossy(&self.buf[..]);
...
}
and remove self.buf = s.into_bytes();
line as you're 1. cleaning it on next line; 2. it was not consumed, but referenced.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With