How do I cancel and rollback a custom action in VS2010 Windows Installer?

Question

I have a custom action that adds/removes a certificate from the Trusted Root Certificates via the Windows Installer. I achieve this by using a CustomAction

It's possible that the user may not have permissions to add the certificate to TrustedRoots, or they may select "Cancel", how do I roll back the previous actions, and tell the installer that I've cancelled the process?

As it stands now the Windows Installer is always reporting a success response even if it fails.

Answer 1

You should set up your custom action to the a function with return type of ActionResult that way you can return the failure type if the cancel happens or another exception.

using Microsoft.Deployment.WindowsInstaller;
namespace CustomAction1
{
    public class CustomActions
    {
        [CustomAction]
        public static ActionResult ActionName(Session session)
        {
            try
            {
                session.Log("Custom Action beginning");

                // Do Stuff...
                if (cancel)
                {
                    session.Log("Custom Action cancelled");
                    return ActionResult.Failure;
                }

                session.Log("Custom Action completed successfully");
                return ActionResult.Success;
            }
            catch (SecurityException ex)
            {
                session.Log("Custom Action failed with following exception: " + ex.Message);
                return ActionResult.Failure;
            }
         }
    }
}

NOTE: This is a WIX compatible custom action. I find WIX to allow for more control over the MSI creation.