How can I switch between fstream files without closing them (Simultaneous output files) - C++

Question

I have a little C++ issue that I couldn't solve by browsing online. Here is my code (extracted):

if(File.is_open()) {
    while(!File.eof())  {
        i++;
        getline(File,Line);
        if(i>=2) {                //Skip Headers
            int CharCount=0;
            for(int CharPosition=0; CharPosition<Line.size(); CharPosition++)                       {
                if(Line[CharPosition]==',') {
                    Length=CharPosition;
                    break;
                }
            }
            NameText=Line.substr(0,Length);
            Path= Path_Folder + "\\" + NameText + ".csv";
            if(!CheckExistance(Path.c_str())) {
                fstream Text_File;
            }
            Text_File.open(Path, fstream::in | fstream::out | fstream::app);
            Text_File<<Line<<"\n";
            Text_File.close();
        }
    }
}

This code is working fine, but I would like to change the fact that it closes the Text_File every time it goes in the while loop.

Basically, this program split a big input file in a lot of smaller files. As my smaller files get bigger and bigger, the execution gets slower and slower (normal). My goal is then to let all the smaller files (Text_File) opened in this while loop and just switch the fstream pointer (pointer?) from one to another.

I tried to change as:

...

NameText=Line.substr(0,Length);
Path= Path_Folder + "\\" + NameText + ".csv";

if(!CheckExistance(Path.c_str())) {
    fstream Text_File;
}

if(!Text_File.open()) {
    Text_File.open(Path, fstream::in |fstream::out | fstream::app);
}

Text_File<<Line<<"\n";
\\Text_File.close();

...

But it is working on the same Text_File no matter what NameText is. So I am guessing that the pointer of the fstream Text_File doesn't change. What do I need to be then? Rest the pointer? How?

Thank you, all!

Not sure it is relevant but I am working with Microsoft Visual C++ 2010 Express. In addition, I am not a programmer neither by education nor by living, so if you can explain it without too advanced words, I'll appreciate.

Answer 1

It looks like you would like to juggle the filebufs on an ostream object.

Now, the only obstacle is that ostream or basic_filebuf<char> aren't copyable types, so you can't put them into a map (by filename) directly. This is easily worked around by creating a little Holder type:

struct Holder {
    Holder(std::string const& path) 
        : buf(std::make_shared<std::filebuf>())
    { 
        buf->open(path.c_str(), std::ios::out | std::ios::app);
    }
    std::shared_ptr<std::filebuf> buf;
};

std::map<std::string, Holder> buffers;

Now the complete program (tested) would look like this:

#include <fstream>
#include <sstream>
#include <iostream>
#include <map>
#include <memory>

const std::string Path_Folder = ".";

int main()
{
    std::istream& File     = std::cin; // just for example
    std::filebuf  dummy;
    std::ostream  TextFile(&dummy);

    struct Holder {
        Holder(std::string const& path) 
            : buf(std::make_shared<std::filebuf>())
        { 
            buf->open(path.c_str(), std::ios::out | std::ios::app);
        }
        std::shared_ptr<std::filebuf> buf;
    };

    std::map<std::string, Holder> buffers;
    int i = 0;

    std::string   Line;
    while(getline(File, Line))
    {
        if (i++<2)
            continue; //Skip Headers

        auto NameText = Line.substr(0, Line.find(','));
        auto Path = Path_Folder + '/' + NameText + ".csv";

        // open, only if not allready opened
        auto found = buffers.find(NameText);
        if (end(buffers) == found)
            found = buffers.insert({ NameText, Path }).first;

        TextFile.rdbuf(found->second.buf.get());

        TextFile << Line << std::endl; // notice implicit std::flush in std::endl
    }

    // all files are automatically closed here
}

Three more notes:

• files get automatically closed when the buffers map goes out of scope.
• you might need to add explicit flushes when switching rdbuf() like this, if you don't end your lines with an implicit std::flush (like with std::endl).
• dummy only exists to have an ostream object that we can switch the buffer of

I tested this with the following input:

Header Row #1
Header Row #2
Jack,1,some data
Jill,2,some more data
Jack,3,not reopening :)
Jill,4,jill still receiving output
Romeo,5,someone else reporting

Now, I got the following output: see it live at Coliru

/tmp$ rm *.csv
/tmp$ make && ./test < input.txt && tail *.csv

g++ -std=c++11 -Wall -g test.cpp -o test
==> Jack.csv <==
Jack,1,some data
Jack,3,not reopening :)

==> Jill.csv <==
Jill,2,some more data
Jill,4,jill still receiving output

==> Romeo.csv <==
Romeo,5,someone else reporting