Custom Action in C# used via WiX fails with error 1154

Question

I am using WiX 3.5.1930 in Visual Studio 2010, targeting the .NET Framework 3.5. (Later weekly builds of WiX seem to be very broken with respect to their custom action template, at least for now. 1930 is the most recent build that seems to make a buildable C# CA with working references.)

I have two custom action assemblies written in C#. One of them works fine. The other fails with the following error:

CustomActionnNameHere returned actual error code 1154 (note this may not be 100% accurate if translation happened inside sandbox)

I have compared the .csproj files and .wixproj files, and as best I can tell the differences are appropriate (e. g. list of included .cs files). I have changed the non-working .wxs to call the working custom action instead of the non-working custom action and it works as epxected.

What else can I look at to get this working?

Edit: Just to be complete 1154 refers to an invalid DLL - net helpmsg translates it (in English) to "One of the library files needed to run this application is damaged."

Second edit: ran peverify against the dll (grabbed a copy out of \windows\installer while the installer was running) and it says everything is fine in the dll. The DLL only has the custom action method with a "return success" so there's not a lot for it to verify, but it does confirm the DLL is not corrupt.

Third edit: The code in the broken custom action follows:

using Microsoft.Deployment.WindowsInstaller;

namespace Framework.Installer.Database {
    public class CustomActions {

        [CustomAction]
        public static ActionResult RunMigration(Session session) {

            return ActionResult.Success;
        }

    }
}

Not much to it. The relevant parts of the .wxs are as follows:

<InstallExecuteSequence>
  <Custom Action="DotNetMigratorCustomActionPreviousUp" After="SetMigrationPropertiesPreviousUp"><![CDATA[(&Database = 3)]]></Custom>
</InstallExecuteSequence>

<Binary Id="DotNetMigratorCustomActionDll"
        SourceFile="$(var.Framework.Installer.Database.CustomActions.TargetDir)\SoftwareAnswers.Framework.Installer.Database.CustomActions.dll" />

<CustomAction Id="DotNetMigratorCustomActionPreviousUp"
              Return="check"
              BinaryKey="DotNetMigratorCustomActionDll"
              DllEntry="RunMigration"
              Execute="deferred" />

Answer 1

If you create your custom action in Visual Studio (Votive) be sure that you created a Wix Custon Action project and not a class library, otherwise you have to use MakeSfxCA tool to pack your custom action.

Answer 2

It sounds like you are using DTF. If you see:

using Microsoft.Deployment.WindowsInstaller;

then you certainly are. Be sure to read the following for how it all works:

Deployment Tools Foundation (DTF) Managed Custom Actions

Also you'll find a DTF help chm in the start menu under WiX.

Basically it sounds like to me you are wiring the .NET assembly into the installer instead of the unmanged wrapper dll. Read the above article for an overview of how to look at it in Depends and to know what to expect. The WiX | C# Custom Action project should output Foo.dll and Foo.CA.dll. You want the later in your installer.

For people who land on this page in the future (the answer was originally for the poster ) there is a whole list of things to check:

1. Are you referencing the correct DLL in the Binary table?
2. Are you referencing the correct exported function name?
3. Is your class public?
4. Is your method using the correct signature? I.e. is it:
5. Marked with the correct CustomAction attribute
6. Marked as public?
7. Marked as static?
8. Return ActionResult?
9. Take Session as an Argument?
10. Make sure you are using the WiX C# Custom Action Project type to ensure the postbuild event is called to create the native DLL wrapper. (See #1)

Any one of these can cause an 1154 error. This is the reason I wrote a comprehensive blog article on the subject and linked to it in this answer. It's important to fully understand how managed code is presented to the unmanaged Windows Installer service and to know how to use Depends to validate that the public static method is exported as a stdcall function in the resulting .CA.dll that WiX/DTF produces.