How can I compile asm shader to fxo file?

Question

I have a compiled fxo shader which I'm trying to edit slightly (just adjusting some constants).

Using fxdis (https://code.google.com/archive/p/fxdis-d3d1x/) I can disassemble this shader, this is the output:

# DXBC chunk  0: RDEF offset 52 size 972
# DXBC chunk  1: ISGN offset 1032 size 80
# DXBC chunk  2: OSGN offset 1120 size 44
# DXBC chunk  3: SHEX offset 1172 size 1592
# DXBC chunk  4: STAT offset 2772 size 148
ps_5_0
dcl_global_flags refactoringAllowed
dcl_constant_buffer cb0[30].xyzw, immediateIndexed
dcl_sampler sampler[0]
dcl_resource_texture2d resource[0]
dcl_resource_texture2d resource[1]
dcl_resource_texture2d resource[2]
dcl_input_ps linear v1.xy
dcl_output o0.xyzw
dcl_temps 5
ne r0.x, l(0, 0, 0, 0), cb0[29].y
mov r1.x, v1.x
add r2.xz, -v1.yyxy, l(1, 0, 1, 0)
eq r0.yzw, cb0[29].yyxz, l(0, 1, 1, 1)
movc r1.y, r0.y, r2.x, v1.y
add r2.y, -r1.y, l(1)
movc r0.xy, r0.xxxx, r2.yzyy, r1.xyxx
sample r1.xyzw, r0.xyxx, resource[1].xyzw, sampler[0]
sample r2.xyzw, r0.xyxx, resource[2].xyzw, sampler[0]
mad r0.xy, r1.zxzz, l(255, 255, 0, 0), r1.wyww
mad r0.xy, r0.xyxx, l(0.0078125, 0.0078125, 0, 0), l(-1, -1, 0, 0)
mul r0.y, r0.y, cb0[28].z
mad r1.x, -r0.x, cb0[28].y, l(1)
add r0.x, r0.x, cb0[28].y
div r0.xy, r0.xyxx, r1.xxxx
ge r1.x, l(1), r0.y
ge r1.y, r0.y, l(-1)
div r0.y, r0.y, cb0[28].x
and r1.x, r1.y, r1.x
ge r1.y, l(1), r0.x
and r1.x, r1.y, r1.x
ge r1.y, r0.x, l(-1)
div r0.x, r0.x, cb0[28].x
add r0.xy, r0.xyxx, l(1, 1, 0, 0)
and r1.x, r1.y, r1.x
and r1.x, r1.x, l(1065353216)
mul r1.y, r0.x, l(0.5)
mad r0.x, -r0.x, l(0.5), l(1)
movc r0.x, r0.z, r1.y, r0.x
add r0.z, -r0.x, l(1)
movc r3.y, r0.w, r0.z, r0.x
mul r3.x, r0.y, l(0.5)
add r3.x, r3.x, l(0.5)
sample r4.xyzw, r3.xyxx, resource[0].xyzw, sampler[0]
mul r1.xyzw, r1.xxxx, r4.xyzw
mul r1.xyzw, r2.xyzw, r1.xyzw
ge r0.x, l(2), r0.y
ge r0.y, r0.y, l(0)
and r0.x, r0.y, r0.x
ge r0.y, r3.y, l(0)
ge r0.z, l(1), r3.y
and r0.x, r0.y, r0.x
and r0.x, r0.z, r0.x
and r0.x, r0.x, l(1065353216)
mul o0.xyzw, r0.xxxx, r1.xyzw
ret

I understand the assembly and can edit it. The problem is, I currently don't know how I can compile this back into an fxo file. It seems fxc.exe from the Windows SDK does not accept asm shader files. Or am I overlooking something?

Answer 1

Creating shaders from assembly is not supported. There were old DirectX assembly tools VSA and PSA in the legacy DirectX SDKs for Shader Model 1.1 - 3.0. They were last shipped in the DirectX SDK (November 2008), well before the end-of-life DirectX SDK (June 2010).

In addition for Shader Model 4.x and 5.x, the DirectX Runtime will not accept a shader blob that is not signed with the correct hash, so binary edits of a shader blob will fail to load. Only the official HLSL compiler can generate a valid shader blob for DirectX 10.x or DirectX 11.

Diassembler output is supported and the shader assembly language is documented on MSDN to facilitate optimization of HLSL shaders, debugging the HLSL compiler itself, and to help with people debugging shaders in applications.

For DirectX 12 and the new Shader Model 6.x tools, see GitHub

Answer 2

Creating bytecode from assembly is not officially supported, but there are resources about opcodes and file format.

This should be a good link to get started : parsing bytecode

Rebuilding bytecode by changing operands is a pretty tedious task, but in your case, if you only require to change a few constants, you can likely use parsing info to replace constant binary value directly (in your case you do not need to rebuild the chunks as replacing a constant does not add/remove new instructions).

In your scenario, you need to go in the SHDR chunk (shader chunk), first find the operand containing the constant you want to modify, for example :

add r2.y, -r1.y, l(1)

and modify the binary value of l(1) by l(want you want)

There is a description of operands in Windows Driver Kit (file is d3d11TokenizedProgramFormat.hpp)

You will also need to modify the hash/checksum that is generated in dxbc, otherwise calling

device->CreateXShader 

will fail.

Checksum is described here

Of course this is not official and supported, so could be a bit error prone.

As mentioned by Chuck, dx12 also have a new open source llvm based compiler which would make those operations much easier, but it's a bit out of your scope since shaders you want to modify are already dxbc (and seems there are no plans to have dx11 dxil support, which is a pity).