Parallel I/O SSD vs HDD surprising results

Question

I have a very strange situation happening with some of my tests regarding paralell I/O. Here is the situation.. I have multiple threads opening a file handler to the same file and reading from multiple locations of the file (evenly spaced intervals) a finite number of bytes and dumping that into an array. All is done with boost threads. Now, I assume with an HDD that should be slower due to the random access seeking. This is why my tests are in fact targeted towards SSD. Turns out I almost do not get any speedup when reading the same file from a solid state disk compared to a HDD. Wonder what the problem might be? Does that seem very surprising just to me / I am also posting my code below to see what I am exactly doing:

    void readFunctor(std::string pathToFile, size_t filePos, BYTE* buffer, size_t buffPos, size_t dataLn, boost::barrier& barier) {

        FILE* pFile;
        pFile = fopen(pathToFile.c_str(), "rb");

        fseek(pFile, filePos, SEEK_SET);
        fread(buffer, sizeof(BYTE), dataLn, pFile);

        fclose(pFile);
        barier.wait();

    }

    void joinAllThreads(std::vector<boost::shared_ptr<boost::thread> > &threads) {

        for (std::vector<boost::shared_ptr<boost::thread> >::iterator it = threads.begin(); it != threads.end(); ++it) {
            (*it).get()->join();

        }

    }

    void readDataInParallel(BYTE* buffer, std::string pathToFile, size_t lenOfData, size_t numThreads) {
        std::vector<boost::shared_ptr<boost::thread> > threads;
        boost::barrier barier(numThreads);
        size_t dataPerThread = lenOfData / numThreads;

        for (int var = 0; var < numThreads; ++var) {
            size_t filePos = var * dataPerThread;
            size_t bufferPos = var * dataPerThread;
            size_t dataLenForCurrentThread = dataPerThread;
            if (var == numThreads - 1) {
                dataLenForCurrentThread = dataLenForCurrentThread + (lenOfData % numThreads);
            }

            boost::shared_ptr<boost::thread> thread(
                    new boost::thread(readFunctor, pathToFile, filePos, buffer, bufferPos, dataLenForCurrentThread, boost::ref(barier)));
            threads.push_back(thread);

        }

        joinAllThreads(threads);

    }

Now.. in my main file I pretty much have..:

    int start_s = clock();
    size_t sizeOfData = 2032221073;
    boost::shared_ptr<BYTE> buffer((BYTE*) malloc(sizeOfData));
    readDataInParallel(buffer.get(), "/home/zahari/Desktop/kernels_big.dat", sizeOfData, 4);
    clock_t stop_s = clock();
    printf("%f %f\n", ((double) start_s / (CLOCKS_PER_SEC)) * 1000, (stop_s / double(CLOCKS_PER_SEC)) * 1000);

Surprisingly, when reading from SSD, I do not get any speedup compared to HDD? Why might that be?

Answer 1

Your file probably gets cached so what you're measuring is the CPU overhead and not the I/O. Instead of flushing the entire disk cache, you can call posix_fadvise() on the file prior to reading it with the "wontneed" flag to advise the kernel not to cache it. That is, assuming you're on some kinda of a *nix platform or Mac OS.

Answer 2

A possible explanation to this is that you are not running under a SATA III setup. The SATA III 6gb/s SSD drive you are using is attached to an older SATA II 3gb/s controller on the motherboard. In that case, your SSD is throttled down to 3 gb/s.

Check your hardware configuration. If it is SATA II, you need to replace the mobo to let your SSD reach its full performance potential.

Check your HDD disk drive also to see if it is SATA, SATA II or SATA III.

Make sure you are comparing apples to apples at the hardware interface level.