Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Qt: How can I copy a big data using QT?

Tags:

c++

qt

I want to read a big data, and then write it to a new file using Qt.

I have tried to read a big file. And the big file only have one line. I test with readAll() and readLine().

If the data file is about 600MB, my code can run although it is slow.

If the data file is about 6GB, my code will fail.

Can you give me some suggestions?

Update
My test code is as following:

#include <QApplication>
#include <QFile>
#include <QTextStream>
#include <QTime>
#include <QDebug>
#define qcout qDebug()

void testFile07()
{
    QFile inFile("../03_testFile/file/bigdata03.txt");
    if (!inFile.open(QIODevice::ReadOnly | QIODevice::Text))
    {
        qcout << inFile.errorString();
        return ;
    }

    QFile outFile("../bigdata-read-02.txt");
    if (!outFile.open(QIODevice::WriteOnly | QIODevice::Truncate))
        return;

    QTime time1, time2;
    time1 = QTime::currentTime();
    while(!inFile.atEnd())
    {
        QByteArray arr = inFile.read(3*1024);
        outFile.write(arr);
    }
    time2 = QTime::currentTime();
    qcout << time1.msecsTo(time2);
}

void testFile08()
{
    QFile inFile("../03_testFile/file/bigdata03.txt");
    if (!inFile.open(QIODevice::ReadOnly | QIODevice::Text))
        return;

    QFile outFile("../bigdata-readall-02.txt");
    if (!outFile.open(QIODevice::WriteOnly | QIODevice::Truncate))
        return;

    QTime time1, time2, time3;
    time1 = QTime::currentTime();

    QByteArray arr = inFile.readAll();
    qcout << arr.size();
    time3 = QTime::currentTime();
    outFile.write(inFile.readAll());

    time2 = QTime::currentTime();
    qcout << time1.msecsTo(time2);

}

int main(int argc, char *argv[])
{
    testFile07();
    testFile08();

    return 0;
}

After my test, I share my experience about it.

  • read() and readAll() seem to be the same fast; more actually, read() is slightly faster.
  • The true difference is writing.

The size of file is 600MB:
Using read function, read and write the file cost about 2.1s, with 875ms for reading
Using readAll function, read and write the file cost about 10s, with 907ms for reading

The size of file is 6GB:
Using read function, read and write the file cost about 162s, with 58s for reading
Using readAll function, get the wrong answer 0. Fail to run well.

like image 621
JosanSun Avatar asked Jan 25 '26 10:01

JosanSun


1 Answers

Open both files as QFiles. In a loop, read a fixed number of bytes, say 4K, into an array from the input file, then write that array into the output file. Continue until you run out of bytes.

However, if you just want to copy a file verbatim, you can use QFile::copy

like image 106
Paul Belanger Avatar answered Jan 27 '26 01:01

Paul Belanger



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!