Why do input attachments need a descriptor set to be bound?

Question

VkRenderPassCreateInfo contains the attachment indices used for Depth, Color and Input attachments. The corresponding image views are referenced in VkFramebufferCreateInfo::pAttachments.

In the shader input_attachment_index identifies which input attachment the shader is using.

Given that the framebuffer is bound during rendering, which should allow identification of the image view to be used for the input attachment, the same way it does for depth and color attachments.

I do not understand why Vulkan also requires mentioning the input attachment (but not other attachments) in descriptor set layouts and consequently in the bound descriptor set.

I feel like I am missing something here? What is it about input attachments that necessitates binding descriptors?

Answer 1

It has to do with pipeline layouts and compatibility.

As it currently stands, pipeline layouts are defined entirely by VkPipelineLayoutCreateInfo. This structure contains some flags, an array of VkDescriptorSetLayout, and an array of VkPushConstantRange. Which means that the layout for a pipeline doesn't actually care about what is defined in the shader; it only cares about the descriptors (and push constants). Obviously, the shaders have to match the descriptors, but that's a matter for the shader to deal with.

Vulkan defines descriptor compatibility between two pipelines based on the compatibility of their pipeline layouts. This means that two pipelines that share the same pipeline layouts are compatible, regardless of the contents of the shaders in those pipelines (or any other pipeline state).

So if we remove input attachments from a descriptor set, we now have this conceptual resource that the shader is using which is not a descriptor. For hardware that implements input attachments as a specialized operation (TBRs, where input attachments are just reading from local tile memory), this is fine. For hardware that implements input attachments as actually reading from textures, this is not fine. Why?

Because of the layout compatibility rules. Remember: they're not based on the stuff in the shader; they're only based on what is in the pipeline layout. So if I have two pipelines that use the same layout, they must be compatible. But if one pipeline has a fragment shader that uses an input attachment and the other does not, then how do you implement that to match the layout compatibility rules, while still having this hidden texture around?

Pipeline layouts and descriptor sets map to some sort of resource binding mechanic in the implementation. So the layout defines how to apply the various descriptor sets to the implementation-defined resource binding. If we look on texture bindings as an array of values, set 0's textures would be assigned before set 1's, and set 2's after that, and so forth.

If input attachments are not descriptors, then how does the binding of the input attachment texture (again, for implementations that treat input attachments as textures) to the implementation-defined resource binding range get done?

1. It could be done at the start of the subpass, since the subpass knows all of the input attachments it uses. The input attachment texture(s) would be bound to a specific texture array location(s) and never changed during that subpass. Let's say we have one input attachment and it picks texture array index 0 for where it is bound.

   But that can't happen; layout compatibility rules don't allow it. Pipelines that don't use the input attachment will assume that index 0 is free to be used by descriptor sets. And therefore, they will be incompatible with pipelines that do use the input attachment. But Vulkan doesn't allow them to be incompatible, since compatibility is defined only for the pipeline layouts, not for properties of the pipeline object itself (like the shaders).

2. It could be done by each pipeline binding call. That is, every time you bind a pipeline that uses an input attachment, it also effectively binds the texture to a certain array index. That array index will be different for different pipelines.

   But this can't happen either, again thanks to layout compatibility rules. If I have pipeline A that uses some descriptor sets layouts 0 and 1, and pipeline B that uses the same initial descriptor set layout 0, then Vulkan says that I can bind descriptor sets to sets 0 and 1 (which match the layouts), and I can use either pipeline in any order with those descriptor sets bound. I don't have to do any descriptor set rebinding between pipeline changes.

   If pipeline B uses an input attachment, then in accord with the above rule, it will assign an additional index in the texture array to that input attachment. But that array index might currently be used by descriptor set 1, which pipeline B has no knowledge of. And therefore, binding pipeline B will break descriptor set 1. Which violates the specification.

Pipeline layout compatibility is a highly useful feature of Vulkan. It allows you to know when you need to bind descriptor sets, as well as when you can switch pipelines without having to change sets. These are good things and useful for runtime performance.

And they simply don't work in a world where input attachments are implemented as textures invisibly. Therefore, Vulkan requires you to make them explicit (since being explicit about things is kinda the point of Vulkan). On implementations that don't make them textures, it just ignores those descriptors entirely.

---

It should also be noted that, according to this presentation (PDF), even TBR's can sometimes treat an input attachment as a texture fetch. This happens when subpasses cannot be "fused" together. This would happen when it can't keep the attachment data in a tile between subpasses. This is based on the properties of the render pass itself, so it's determinable up-front.

So that's why it's an input attachment.