Most efficient way of creating a progress bar while reading input from a file

Question

I have a program which is reading in input from a relatively large file, thousands of lines long.

With that being said, I want to implement a progress bar indicator while the file is being processed. However, most methods I'm aware of require you to use getLine to count how many lines there are in the file to use this as the "predefined goal" for your progress bar (Boost Example). That means I'd have to loop through a large text file twice, once to count how many lines, and another to actually get each line and display a progress bar.

Is there a more efficient way?

Answer 1

A possible solution is to seek to the end of the file, just to learn the size of the input. Then, keep updating a progress bar based on the percentage of the file you have already processed. This should give you a very nice and simple progress bar -- which may be prettified with ASCII-art and carriage returns (\r).

Here is also a possible implementation:

# include <cmath>
# include <string>
# include <fstream>
# include <iomanip>
# include <iostream>


class reader : public std::ifstream {

public:

    // Constructor
    template <class... Args>
    inline reader(int max, Args&&... args) :
    std::ifstream(args...), _max(max), _last(0) {
        if (std::ifstream::is_open()) _measure();
    }

    // Opens the file and measures its length
    template <class... Args>
    inline auto open(Args&&... args)
    -> decltype(std::ifstream::open(args...)) {
        auto rvalue(std::ifstream::open(args...));
        if (std::ifstream::is_open()) _measure();
        return rvalue;
    }

    // Displays the progress bar (pos == -1 -> end of file)
    inline void drawbar(void) {

        int pos(std::ifstream::tellg());
        float prog(pos / float(_length)); // percentage of infile already read
        if (pos == -1) { _print(_max + 1, 1); return; }

        // Number of #'s as function of current progress "prog"
        int cur(std::ceil(prog * _max));
        if (_last != cur) _last = cur, _print(cur, prog);

    }

private:

    std::string _inpath;
    int _max, _length, _last;

    // Measures the length of the input file
    inline void _measure(void) {
        std::ifstream::seekg(0, end);
        _length = std::ifstream::tellg();
        std::ifstream::seekg(0, beg);
    }

    // Prints out the progress bar
    inline void _print(int cur, float prog) {

        std::cout << std::fixed << std::setprecision(2)
            << "\r   [" << std::string(cur, '#')
            << std::string(_max + 1 - cur, ' ') << "] " << 100 * prog << "%";

        if (prog == 1) std::cout << std::endl;
        else std::cout.flush();

    }

};


int main(int argc, char *argv[]) {

    // Creating reader with display of length 100 (100 #'s)
    reader infile(std::atoi(argv[2]), argv[1]);
    std::cout << "-- reading file \"" << argv[1] << "\"" << std::endl;

    std::string line;
    while (std::getline(infile, line)) infile.drawbar();

}

And the output is something like:

$ ./reader foo.txt 50              # ./reader <inpath> <num_#'s>
-- reading file "foo.txt"
   [###################################################] 100.00%

Notice the parameters are the input file and the number of #'s you want in your progress bar. I've added the length-seek to std::ifstream::open function, but the drawbar() is called by the user. You may jam this function inside a specific function from std::ifstream.

If you want to get it fancier, you could as well use the command tput cols, to learn the number of columns in your current shell. Also, you may put such command inside the executable, to make it clearer than this:

$ ./reader foo.txt $(( $(tput cols) - 30 ))
-- reading file "foo.txt"
   [####################################################################] 100.00%

As others pointed, this solution does not work properly with pipes and temporary files, in which cases you don't have the input length at hand. Many thanks @NirMH, for the very kind comment.