basic_filebuf::underflow error reading the file with ifstream on /proc/pid/stat

Question

Why does the following code throw an exception? Note that the file is a /proc/pid/stat file so it could be interfered by the kernel.

// Checked that file does exist
try {
  std::ifstream file(path.c_str());
  // Shouldn't even be necessary because it's the default but it doesn't 
  // make any difference.
  file.exceptions(std::ifstream::goodbit);
  // Read the stream into many fields
  // !!!! The exception was thrown here.
  file >> _ >> comm >> state >> ppid >> pgrp >> session >> tty_nr
       /* >> ... omitted */;
  file.close();
} catch (const std::ifstream::failure& e) {
  std::cout << "Exception!!!! " << e.what();
}

The exception was "basic_filebuf::underflow error reading the file".

Shouldn't the stream not throw an exception when we haven't asked it to (by setting file.exceptions())?

More info:

• It runs on gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)
• The full code that causes the problem (without the try/catch part): proc.hpp

Answer 1

Update 2

I've even tried to force an error by manually setting tiny or huge buffer sizes:

    std::filebuf fb;
    // set tiny input buffer
    char buf[8]; // or huge: 64*1024
    fb.pubsetbuf(buf, sizeof(buf));
    fb.open(path.c_str(), std::ios::in);

    std::istream file(&fb);

I've verified that the read size were indeed tiny (7) using strace

sudo strace ./test $(sudo ps h -ae -o pid) |& 
      egrep -w 'read|open' | grep -v '= 7' | less -SR

Interestingly, none of this failed.

Updated

in response to the comments, I have devised a standalone program that does exactly what the OP describes, but I can't reproduce the problem:

• See it live on Coliru again http://coliru.stacked-crooked.com/view?id=81960c...

#include <sys/types.h> // For pid_t.
#include <fstream>
#include <string>

// mock up
#include <boost/variant.hpp>
namespace {
    struct None {};
    struct Error { std::string s; Error(std::string s): s(s){} };

    std::ostream& operator<<(std::ostream& os, None const&) {
        return os << "None";
    }

    std::ostream& operator<<(std::ostream& os, Error const& e) {
        return os << "Error {" << e.s << "}";
    }

    template <typename T>
        using Result = boost::variant<None, Error, T>;
}
// end mockup

namespace proc {

    // Snapshot of a process (modeled after /proc/[pid]/stat).
    // For more information, see:
    // http://www.kernel.org/doc/Documentation/filesystems/proc.txt
    struct ProcessStatus
    {
        pid_t pid;
        std::string comm;
        char state;
        pid_t ppid, pgrp, session;
        int tty_nr;
        pid_t tpgid;
        unsigned int flags;
        unsigned long minflt, cminflt, majflt, cmajflt;
        unsigned long utime, stime;
        long cutime, cstime, priority, nice, num_threads, itrealvalue;
        unsigned long long starttime;
        unsigned long vsize;
        long rss;
        unsigned long rsslim, startcode, endcode, startstack, kstkeip, signal, blocked, sigcatch, wchan, nswap, cnswap;

        friend std::ostream& operator<<(std::ostream& os, proc::ProcessStatus const& ps) {
            return os << 
                "pid: "         << ps.pid         << "\n" << 
                "comm: "        << ps.comm        << "\n" << 
                "state: "       << ps.state       << "\n" << 
                "ppid: "        << ps.ppid        << "\n" << 
                "pgrp: "        << ps.pgrp        << "\n" << 
                "session: "     << ps.session     << "\n" << 
                "tty_nr: "      << ps.tty_nr      << "\n" << 
                "tpgid: "       << ps.tpgid       << "\n" << 
                "flags: "       << ps.flags       << "\n" << 
                "minflt: "      << ps.minflt      << "\n" << 
                "cminflt: "     << ps.cminflt     << "\n" << 
                "majflt: "      << ps.majflt      << "\n" << 
                "cmajflt: "     << ps.cmajflt     << "\n" << 
                "utime: "       << ps.utime       << "\n" << 
                "stime: "       << ps.stime       << "\n" << 
                "cutime: "      << ps.cutime      << "\n" << 
                "cstime: "      << ps.cstime      << "\n" << 
                "priority: "    << ps.priority    << "\n" << 
                "nice: "        << ps.nice        << "\n" << 
                "num_threads: " << ps.num_threads << "\n" << 
                "itrealvalue: " << ps.itrealvalue << "\n" << 
                "starttime: "   << ps.starttime   << "\n" << 
                "vsize: "       << ps.vsize       << "\n" << 
                "rss: "         << ps.rss         << "\n" << 
                "rsslim: "      << ps.rsslim      << "\n" << 
                "startcode: "   << ps.startcode   << "\n" << 
                "endcode: "     << ps.endcode     << "\n" << 
                "startstack: "  << ps.startstack  << "\n" << 
                "kstkeip: "     << ps.kstkeip     << "\n" << 
                "signal: "      << ps.signal      << "\n" << 
                "blocked: "     << ps.blocked     << "\n" << 
                "sigcatch: "    << ps.sigcatch    << "\n" << 
                "wchan: "       << ps.wchan       << "\n" << 
                "nswap: "       << ps.nswap       << "\n" << 
                "cnswap: "      << ps.cnswap      << "\n";
        }
    };

    // Returns the process statistics from /proc/[pid]/stat.
    // The return value is None if the process does not exist.
    inline Result<ProcessStatus> status(pid_t pid)
    {
        std::string path = "/proc/" + std::to_string(pid) + "/stat";

        std::ifstream file(path.c_str());

        if (!file.is_open()) {
#if 1
            return Error("Failed to open '" + path + "'");
#else // FIXME reenable
            // Need to check if file exists AFTER we open it to guarantee
            // process hasn't terminated (or if it has, we at least have a
            // file which the kernel _should_ respect until a close).
            if (!os::exists(path)) {
                return None();
            }
            return Error("Failed to open '" + path + "'");
#endif
        }

        std::string _; // For ignoring fields.

        // Parse all fields from stat.
        ProcessStatus ps;
        if (file >> _ >> ps.comm >> ps.state >> ps.ppid >> ps.pgrp >> ps.session >> ps.tty_nr
            >> ps.tpgid >> ps.flags >> ps.minflt >> ps.cminflt >> ps.majflt >> ps.cmajflt
            >> ps.utime >> ps.stime >> ps.cutime >> ps.cstime >> ps.priority >> ps.nice
            >> ps.num_threads >> ps.itrealvalue >> ps.starttime >> ps.vsize >> ps.rss
            >> ps.rsslim >> ps.startcode >> ps.endcode >> ps.startstack >> ps.kstkeip
            >> ps.signal >> ps.blocked >> ps.sigcatch >> ps.wchan >> ps.nswap >> ps.cnswap)
        {
            return ps;
        } else
        {
            return Error("Failed to read/parse '" + path + "'");
        }
    }


} // namespace proc {

int main(int argc, const char *argv[])
{
    for (auto i=1; i<argc; ++i)
        std::cout << proc::status(std::stoul(argv[i])) << "\n";
}

It runs happily on my machine, printing stuff like

pid: 594590200
comm: (test)
state: R
ppid: 8123
pgrp: 8123
session: 8123
...

Even if/when I torture it with

sudo ./test $(sudo ps h -ae -o pid) | grep -v : | sort -u
./test $(sudo ps h -ae -o pid) | grep -v : | sort -u

It just shows (presumably the sudo/ps from the subshell)

Error {Failed to open '/proc/8652/stat'}
Error {Failed to open '/proc/8653/stat'}

I have tried to read the information twice from the input stream (to force read-past-the-end type situation), but no luck.

---

Old answer text:

You need to establish under what kind of conditions the exception occurs. The following demo code works as expected on my system:

#include <fstream>
#include <iostream>

int main()
{
    system("ps -o pid,comm,state,ppid,pgrp,session,tty > input.txt");

    try {
        std::ifstream file("input.txt");

        file.exceptions(std::ifstream::goodbit);

        std::string _, comm, state, ppid, pgrp, session, tty_nr;
        while (file >> _ >> comm >> state >> ppid >> pgrp >> session >> tty_nr)
        {
            for (auto&& s : { _, comm, state, ppid, pgrp, session, tty_nr })
                std::cout << s << "\t";
            std::cout << "\n";
        }
        file.close();
    } catch (const std::ifstream::failure& e) {
        std::cout << "Exception!!!! " << e.what();
    }
}

Prints, e.g.:

PID         COMMAND S       PPID    PGRP    SESS    TT      
20950       bash    S       20945   20950   20950   pts/1   
21275       vim     S       20950   21275   20950   pts/1   
21279       bash    S       21275   21275   20950   pts/1   
21280       test    S       21279   21275   20950   pts/1   
21281       sh      S       21280   21275   20950   pts/1   
21282       ps      R       21281   21275   20950   pts/1   

Update

I thought, on many systems the default buffer size is probably max 8192 bytes; let's create some silly long lines instead! Replacing the system call by

system("od /dev/urandom -t x8 -Anone | xargs -n256 | tr -d ' ' | xargs -n7 | head -n100 > input.txt");

results in lines with 7 columns, taking ~29kB per line. The output is without hesitation and amounts to 2.8MiB of output, measured with

make -B  && ./test | wc 

See it live on coliru:

• http://coliru.stacked-crooked.com/view?id=00160c8a2b5d4447cbda3866da3913aa-b8c04540b49ee47c13fd3df477ec213b

Note that on Coliru, we can't access /dev/urandom, which is why I read from the binary itself. "Random" enough for this purpose :)