How to make QVideoProbe work?

Question

I'm currently making a cross-platform app (Android, iOS) using QML, and I need to do a QR code scanning function. The QR code readings are using ZXing and there is no problem, the problem is on the camera. I can't extract video frames from QCamera.

There is a module QVideoProbe which is supposed to do it for me. However, it doesn't work , not on Win32, not on OSX. It simply doesn't work on desktop platform. I don't have my code on this computer, but it's just like this example I found on the forum

QCamera *camera = new QCamera;
camera->setCaptureMode(QCamera::CaptureVideo);
QCameraViewfinder *viewfinder = new QCameraViewfinder();
camera->setViewfinder(viewfinder);
qDebug() << " start set source";
QVideoProbe *videoProbe = new QVideoProbe(this);
if (videoProbe->setSource((QMediaObject *)camera)) {
    qDebug() << " set source succeed";
    //Probing succeeded, videoProbe->isValid() should be true.
    connect(videoProbe, SIGNAL(videoFrameProbed(const QVideoFrame &)),this,SLOT(detectAVA(const QVideoFrame &)));
}
camera->start();

The guy on the forum has the same problem with me. The line:

videoProbe->setSource((QMediaObject *)camera)

will return false.

The reply on the forum is:

Using (passing) a Camera element as sourceObj, what will happen?

It will also work.

That was in theory. In practice, it depends on the platform. QVideoProbe is not available on all of them (or only for either a media player or a camera).

It should be in the doc, but here's an overview of QVideoProbe support:

• Android: only for camera
• Blackberry: no support
• iOS: no support
• Linux: only for media player
• Mac: no support
• Windows: only for media player

If it’s not supported it doesn't necessarily mean it's not possible to do it on a given platform, it can mean that it’s not currently implemented.

So, Android is only platform that support for Camera, then I am looking for support on Win32, I am willing to add support under win32, is that work will be really hard, and I will merge it into Qt mainline.

Besides, I will add media player support under Android. That’s two feature I am looking for.

Looks like it's not implemented, and if this works, it will only work on Android, and I have to test it on a real phone (simulator doesn't work).

I guess extracting a frame from camera is a very basic function on any platform and language. There must be a way. Is there any solution?

Answer 1

I implemented this using a hack. It works somewhat as expected. The idea is to use QAbstractVideoSurface as a makeshift alternative to QVideoProbe. It has many drawbacks, but at least it is something.

Code below compiles but I honestly don't know if it works (it has been sitting dormant for a while). Use at your own risk!

PoorMansProbe.hpp

#ifndef POORMANSPROBE_HPP
#define POORMANSPROBE_HPP

#include <QAbstractVideoSurface>
#include <QList>

class QCamera;
class QCameraViewfinder;

class PoorMansProbe : public QAbstractVideoSurface
{
    Q_OBJECT

private:
    QCamera *source;
public:
    explicit PoorMansProbe(QObject *parent = nullptr);

    QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const;

    // Called from QAbstractVideoSurface whenever a new frame is present
    bool present(const QVideoFrame &frame) Q_DECL_OVERRIDE;

    bool setSource(QCamera *source);

    bool isActive() const;


signals:
    // Users of this class will get frames via this signal
    void videoFrameProbed(const QVideoFrame &videoFrame);
    void flush();

};

#endif // POORMANSPROBE_HPP

PoorMansProbe.cpp

#include "PoorMansProbe.hpp"

#include <QVideoFrame>
#include <QCamera>
#include <QCameraViewfinder>


PoorMansProbe::PoorMansProbe(QObject *parent)
    : QAbstractVideoSurface(parent)
    , source(0)
{
}

QList<QVideoFrame::PixelFormat> PoorMansProbe::supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
    Q_UNUSED(handleType);
    return QList<QVideoFrame::PixelFormat>()
           << QVideoFrame::Format_ARGB32
           << QVideoFrame::Format_ARGB32_Premultiplied
           << QVideoFrame::Format_RGB32
           << QVideoFrame::Format_RGB24
           << QVideoFrame::Format_RGB565
           << QVideoFrame::Format_RGB555
           << QVideoFrame::Format_ARGB8565_Premultiplied
           << QVideoFrame::Format_BGRA32
           << QVideoFrame::Format_BGRA32_Premultiplied
           << QVideoFrame::Format_BGR32
           << QVideoFrame::Format_BGR24
           << QVideoFrame::Format_BGR565
           << QVideoFrame::Format_BGR555
           << QVideoFrame::Format_BGRA5658_Premultiplied
           << QVideoFrame::Format_AYUV444
           << QVideoFrame::Format_AYUV444_Premultiplied
           << QVideoFrame::Format_YUV444
           << QVideoFrame::Format_YUV420P
           << QVideoFrame::Format_YV12
           << QVideoFrame::Format_UYVY
           << QVideoFrame::Format_YUYV
           << QVideoFrame::Format_NV12
           << QVideoFrame::Format_NV21
           << QVideoFrame::Format_IMC1
           << QVideoFrame::Format_IMC2
           << QVideoFrame::Format_IMC3
           << QVideoFrame::Format_IMC4
           << QVideoFrame::Format_Y8
           << QVideoFrame::Format_Y16
           << QVideoFrame::Format_Jpeg
           << QVideoFrame::Format_CameraRaw
           << QVideoFrame::Format_AdobeDng;
}

bool PoorMansProbe::present(const QVideoFrame &frame)
{
    if (frame.isValid()) {
        emit videoFrameProbed(frame);
        return true;
    }
    return false;
}

bool PoorMansProbe::setSource(QCamera *source)
{
    this->source=source;
    source->setViewfinder(this);
    QCameraViewfinderSettings viewfinderSettings;
    // I have a broken camera that only support these settings, but you should put whatever settings you need here
    viewfinderSettings.setResolution(640, 480);
    viewfinderSettings.setMinimumFrameRate(0.0);
    viewfinderSettings.setMaximumFrameRate(30.0);
    source->setViewfinderSettings(viewfinderSettings);
    return true;
}


bool PoorMansProbe::isActive() const
{
    return (0!=source);
}