How to combine std::str::lines and std::io::lines?

Question

I want to write a function to parse text, but the text may come from external file or an internal &str. The parse function may go like this:

fn parse(lines: GenericLinesGenerator) {
   for line in lines {
       // parse content
   }
}

... and it can be invoked like this:

use std::io::BufReader;
use std::fs::File;
let fin = BufReader::new(File::open("info.txt").expect("not found"));
parse(TransformFromIO(fin.lines()))

or

let content: &'static str = "some\nlong\ntext";
parse(TransformFromStr(content.lines()))

Is it possible to implement such a parse function?

Answer 1

The two iterators don't produce the same values:

impl<B: BufRead> Iterator for io::Lines<B> {
    type Item = Result<String>;
}

impl<'a> Iterator for str::Lines<'a> {
    type Item = &'a str;
}

You have to handle that difference somehow. The most important difference is that io::Lines can fail. Your program has to decide how to deal with that; I've chosen to just abort the program.

The next thing you need to do is to accept any type that can be converted into an iterator, and the value yielded from the iterator has to be converted to a type that you can deal with. It appears that &str is the common denominator.

This is solved by using IntoIterator and Borrow:

use std::borrow::Borrow;
use std::fs::File;
use std::io::prelude::*;
use std::io::BufReader;

fn parse<I>(lines: I)
where
    I: IntoIterator,
    I::Item: Borrow<str>,
{
    for line in lines {
        println!("line: {}", line.borrow());
    }
}

fn main() {
    parse("alpha\nbeta\ngamma".lines());

    println!("----");

    let f = File::open("/etc/hosts").expect("Couldn't open");
    let b = BufReader::new(f);
    parse(b.lines().map(|l| l.expect("Bad line!")));
}

Check The Rust Programming Language section on where clauses for more information about trait bounds.

Answer 2

Using the Borrow bound in the parse function will allow you to borrow &str, but if you need String values, a better approach is to use Cow.

Using line.borrow().to_string() to obtain a String value will always allocate, even when parse is called with lines from file (in this case, lines.map produces String).

Using line.into_owned() will allocate when called with lines from an &str, but will not allocate when called with lines from file (will just unwrap the String value passed to Cow::Owned).

use std::borrow::Cow;
use std::io::{BufReader, BufRead};
use std::iter::IntoIterator;
use std::fs::File;

fn parse<'a, I>(lines: I)
    where I: IntoIterator,
          I::Item: Into<Cow<'a, str>>
{
    for line in lines {
        let line: Cow<'a, str> = line.into();
        let line: String = line.into_owned();
        // or
        let line = line.into().into_owned()
        println!("{}", line);
    }
}

fn main() {
    let fin = BufReader::new(File::open("/etc/hosts").expect("cannot open file"));
    parse(fin.lines().map(|r| r.expect("file read failed")));

    let content: &'static str = "some\nlong\ntext";
    parse(content.lines());
}