Billboarding using Qt3D 2.0

Question

I am looking for the best way to create a billboard in Qt3D. I would like a plane which faces the camera wherever it is and does not change sized when the camera dollies forward or back. I have read how to do this using GLSL vertex and geometry shaders, but I am looking for the Qt3D way, unless customer shaders is the most efficient and best way of billboarding.

I have looked, and it appears I can set the Matrix on a QTransform via properties, but it isn't clear to me how I would manipulate the matrix, or perhaps there is a better way? I am using the C++ api, but a QML answer would do. I could port it to C++.

Answer 1

If you want to draw just one billboard, you can add a plane and rotate it whenever the camera moves. However, if you want to do this efficiently with thousands or millions of billboards, I recommend using custom shaders. We did this to draw impostor spheres in Qt3D.

However, we didn't use a geometry shader because we were targeting systems that didn't support geometry shaders. Instead, we used only the vertex shader by placing four vertices in the origin and moved these on the shader. To create many copies, we used instanced drawing. We moved each set of four vertices according to the positions of the spheres. Finally, we moved each of the four vertices of each sphere such that they result in a billboard that is always facing the camera.

Start out by subclassing QGeometry and created a buffer functor that creates four points, all in the origin (see spherespointgeometry.cpp). Give each point an ID that we can use later. If you use geometry shaders, the ID is not needed and you can get away with creating only one vertex.

class SpheresPointVertexDataFunctor : public Qt3DRender::QBufferDataGenerator
{
public:
    SpheresPointVertexDataFunctor()
    {
    }

    QByteArray operator ()() Q_DECL_OVERRIDE
    {
        const int verticesCount = 4;
        // vec3 pos
        const quint32 vertexSize = (3+1) * sizeof(float);

        QByteArray verticesData;
        verticesData.resize(vertexSize*verticesCount);
        float *verticesPtr = reinterpret_cast<float*>(verticesData.data());

        // Vertex 1
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        // VertexID 1
        *verticesPtr++ = 0.0;

        // Vertex 2
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        // VertexID 2
        *verticesPtr++ = 1.0;

        // Vertex 3
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        // VertexID3
        *verticesPtr++ = 2.0;

        // Vertex 4
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        *verticesPtr++ = 0.0;
        // VertexID 4
        *verticesPtr++ = 3.0;

        return verticesData;
    }

    bool operator ==(const QBufferDataGenerator &other) const Q_DECL_OVERRIDE
    {
        Q_UNUSED(other);
        return true;
    }

    QT3D_FUNCTOR(SpheresPointVertexDataFunctor)
};

For the real positions, we used a separate QBuffer. We also set color and scale, but I have omitted those here (see spheredata.cpp):

void SphereData::setPositions(QVector<QVector3D> positions, QVector3D color, float scale)
{
    QByteArray ba;
    ba.resize(positions.size() * sizeof(QVector3D));
    SphereVBOData *vboData = reinterpret_cast<QVector3D *>(ba.data());
    for(int i=0; i<positions.size(); i++) {
        QVector3D &position = vboData[i];
        position = positions[i];
    }
    m_buffer->setData(ba);
    m_count = positions.count();
}

Then, in QML, we connected the geometry with the buffer in a QGeometryRenderer. This can also be done in C++, if you prefer (see Spheres.qml):

GeometryRenderer {
    id: spheresMeshInstanced
    primitiveType: GeometryRenderer.TriangleStrip
    enabled: instanceCount != 0
    instanceCount: sphereData.count

    geometry: SpheresPointGeometry {
        attributes: [
            Attribute {
                name: "pos"
                attributeType: Attribute.VertexAttribute
                vertexBaseType: Attribute.Float
                vertexSize: 3
                byteOffset: 0
                byteStride: (3 + 3 + 1) * 4
                divisor: 1
                buffer: sphereData ? sphereData.buffer : null
            }
        ]
    }
}

Finally, we created custom shaders to draw the billboards. Note that because we were drawing impostor spheres, the billboard size was increased to handle raytracing in the fragment shader from awkward angles. You likely do not need the 2.0*0.6 factor in general.

Vertex shader:

#version 330

in vec3 vertexPosition;
in float vertexId;
in vec3 pos;
in vec3 col;
in float scale;

uniform vec3 eyePosition = vec3(0.0, 0.0, 0.0);

uniform mat4 modelMatrix;
uniform mat4 mvp;

out vec3 modelSpherePosition;
out vec3 modelPosition;
out vec3 color;
out vec2 planePosition;
out float radius;
vec3 makePerpendicular(vec3 v) {
    if(v.x == 0.0 && v.y == 0.0) {
        if(v.z == 0.0) {
            return vec3(0.0, 0.0, 0.0);
        }
        return vec3(0.0, 1.0, 0.0);
    }
    return vec3(-v.y, v.x, 0.0);
}

void main() {
    vec3 position = vertexPosition + pos;
    color = col;
    radius = scale;
    modelSpherePosition = (modelMatrix * vec4(position, 1.0)).xyz;

    vec3 view = normalize(position - eyePosition);
    vec3 right = normalize(makePerpendicular(view));
    vec3 up = cross(right, view);

    float texCoordX = 1.0 - 2.0*(float(vertexId==0.0) + float(vertexId==2.0));
    float texCoordY = 1.0 - 2.0*(float(vertexId==0.0) + float(vertexId==1.0));
    planePosition = vec2(texCoordX, texCoordY);

    position += 2*0.6*(-up - right)*(scale*float(vertexId==0.0));
    position += 2*0.6*(-up + right)*(scale*float(vertexId==1.0));
    position += 2*0.6*(up - right)*(scale*float(vertexId==2.0));
    position += 2*0.6*(up + right)*(scale*float(vertexId==3.0));

    vec4 modelPositionTmp = modelMatrix * vec4(position, 1.0);
    modelPosition = modelPositionTmp.xyz;

    gl_Position = mvp*vec4(position, 1.0);
}

Fragment shader:

#version 330

in vec3 modelPosition;
in vec3 modelSpherePosition;
in vec3 color;
in vec2 planePosition;
in float radius;

out vec4 fragColor;

uniform mat4 modelView;
uniform mat4 inverseModelView;
uniform mat4 inverseViewMatrix;
uniform vec3 eyePosition;
uniform vec3 viewVector;

void main(void) {
    vec3 rayDirection = eyePosition - modelPosition;
    vec3 rayOrigin = modelPosition - modelSpherePosition;

    vec3 E = rayOrigin;
    vec3 D = rayDirection;

    // Sphere equation
    //      x^2 + y^2 + z^2 = r^2
    // Ray equation is
    //     P(t) = E + t*D
    // We substitute ray into sphere equation to get
    //     (Ex + Dx * t)^2 + (Ey + Dy * t)^2 + (Ez + Dz * t)^2 = r^2
    float r2 = radius*radius;
    float a = D.x*D.x + D.y*D.y + D.z*D.z;
    float b = 2.0*E.x*D.x + 2.0*E.y*D.y + 2.0*E.z*D.z;
    float c = E.x*E.x + E.y*E.y + E.z*E.z - r2;

    // discriminant of sphere equation
    float d = b*b - 4.0*a*c;
    if(d < 0.0) {
        discard;
    }

    float t = (-b + sqrt(d))/(2.0*a);
    vec3 sphereIntersection = rayOrigin + t * rayDirection;

    vec3 normal = normalize(sphereIntersection);
    vec3 normalDotCamera = color*dot(normal, normalize(rayDirection));

    float pi = 3.1415926535897932384626433832795;

    vec3 position = modelSpherePosition + sphereIntersection;

    // flat red
    fragColor = vec4(1.0, 0.0, 0.0, 1.0);
}

It has been some time since we first implemented this, and there might be easier ways to do it now, but this should give you an idea of the pieces you need.