Flipping the viewport in Vulkan

Question

From the spec:

VK_KHR_maintenance1

Allow negative height to be specified in the slink::VkViewport::height field to perform y-inversion of the clip-space to framebuffer-space transform. This allows apps to avoid having to use gl_Position.y = -gl_Position.y in shaders also targeting other APIs.

My physical device supports version 1.0.42 and I have enabled the VK_KHR_maintenance1. I do not get any validation error/warning when I do a negative height in the VkViewport struct.

vkCmdSetViewport(vkContext.commandBuffer, 0, 1, &viewport);

However I do not see anything on the screen either, its black, if I remove the negative value in the viewport everything is rendered as expected. Do I need to do anything else to flip the viewport with the VK_KHR_maintenance1 extension? I am just rendering a red quad on the screen, with a fullscreen viewport.

VkViewport viewport = {};
viewport.x = 0.0f;
viewport.y = 0.0f;
viewport.width = 1024.0f;
viewport.height = -1024.0f;
viewport.minDepth = 0.0f;
viewport.maxDepth = 1.0f;
vkCmdSetViewport(vkContext.commandBuffer, 0, 1, &viewport);

Vertex shader:

void main() {
        vec4 positions[3] = {
        vec4(-1.0, -1.0, 0.0, 1.0),
        vec4(3.0, -1.0, 0.0, 1.0),
        vec4(-1.0, 3.0, 0.0, 1.0)
    };
    gl_Position = positions[gl_VertexIndex % 3];
}

Fragment shader:

void main() {
    outFragColor = vec4(1,0,0,1);
}

Answer 1

Just go through the math. You have all of the numbers, and the specification explicitly lays it all out.

The equations for the Y component of the framebuffer-space vertex coordinate are:

Yf = (Py / 2) Yd + Oy
Oy = viewport.y + viewport.height / 2.0
Py = viewport.height

Where Yf is the framebuffer-space Y component and Yd is the NDC-space Y component.

If you plug in a positive height of 1024, what numbers do you get? For a Yd of -1, you get a Yf of 0.0. For a Yd of 3, you get a Yf of 2048.

In Vulkan, the "natural" viewport transform maps NDC-space by putting -1 at the viewport.x/y, with 1 at the viewport.width/height. And remember: Vulkan's images have a top-left origin, with positive values going down and to the right. So the Yf of 0.0 is at the top, and the Yf of 2048 is off the bottom of the screen. Therefore, the triangle is (partly) in the visible area of the view port.

Given that, what happens when you use a negative height? The math doesn't change; just the results. For a Yd of -1, you get a Yf of 0.0, much like before. For a Yd of 3, you get a Yf of -2048.

What does that look like? Well, the meaning of the viewport hasn't changed; just the numbers. That means that the visible area of the framebuffer image is still at the top left, going down. And a value of -2048 is therefore above the viewport.

In short, a negative height flips the viewport. But that flips it at the viewport origin, not at the center of the screen. The viewport origin is at the top-left going down and to the right. And therefore, flipping it will flip everything that was previously visible off of the screen. You're now looking at a different area of the screen, which in your case was blank.

If you want to flip everything around the center of the viewport, then you need to adjust the viewport.y to compensate for the negative height. Specifically, it needs to be the positive height.