Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS CloudFormation stack fails with error Received 0 SUCCESS signal(s) out of 1

My AWS CloudFormation template fails with the error:

Received 0 SUCCESS signal(s) out of 1. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement

AWS Coudformation stack events

I'm thinking my WaitConditionHandles are not set correctly (or maybe the EC2 instance is not sending one), but not sure how to fix this.

Everything (ASG, EC2 instances) does appear to be created correctly in AWS.

I'm using the following CloudFormation template:

AWSTemplateFormatVersion: "2010-09-09"
Description: "Auto Scaling Group"
Outputs:
  AsgArn: 
    Value: !Ref "AutoScalingGroup"
  AsgMinSize:
    Description: "The minimum size of the Auto Scaling Group"
    Value: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
Parameters:
  Ami:
    Description: "Base AMI"
    Type: "AWS::EC2::Image::Id"
  EnvironmentName:
    Description: "The environment name"
    Type: "String"
  HighAvailabilityFlag:
    Description: "Flag used to set the minimum and maximum size of the Auto Scaling Group"
    Default: false
    Type: "String"
    AllowedValues: [ "true", "false" ]
  KeyPairName:
    Description: "Name of EC2 key pair for logging in to the instances"
    Type: "String"
  SecurityGroupIds:
    Description: "The IDs of security groups that are permitted access to EC2 instances"
    Type: "String"
  Subnets:
    Description: "Subnets to associate with the ASG"
    Type: "List<AWS::EC2::Subnet::Id>"
  VersionToDeploy:
    Description: "Version to deploy"
    Type: "String"
  VpcId:
    Description: "The ID of the VPC"
    Type: "AWS::EC2::VPC::Id"
Mappings:
  HighAvailability:
    MinSize:
      "false": 1
      "true": 2
    MaxSize:
      "false": 1
      "true": 4
Resources:
  InstanceProfile:
    Properties:
      Path: "/"
      Roles:
        - !Ref "InstanceRole"
    Type: "AWS::IAM::InstanceProfile"
  InstanceRole:
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action:
              - sts:AssumeRole
            Effect: "Allow"
            Principal:
              Service:
                - ec2.amazonaws.com
        Version: "2012-10-17"
      Path: "/"
    Type: "AWS::IAM::Role"
  Policy:
    Properties:
      PolicyDocument:
        Statement:
          - Action:
              - cloudformation:DescribeStacks
              - ec2:Describe*
            Effect: "Allow"
            Resource: "*"
        Version: "2012-10-17"
      PolicyName: "Service"
      Roles:
        - !Ref "InstanceRole"
    Type: "AWS::IAM::Policy"    
  AutoScalingGroup:
    Properties:
      HealthCheckGracePeriod: 300
      MetricsCollection:
        - Granularity: "1Minute"
      HealthCheckType: "ELB"
      LaunchConfigurationName: !Ref "LaunchConfiguration"
      MaxSize: !FindInMap [ "HighAvailability", "MaxSize", !Ref "HighAvailabilityFlag" ]
      MinSize: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
      VPCZoneIdentifier: !Ref "Subnets"
    CreationPolicy:
      ResourceSignal:
        Count: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
        Timeout: "PT5M"
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: !FindInMap [ "HighAvailability", "MinSize", !Ref "HighAvailabilityFlag" ]
        PauseTime: "PT5M"
        WaitOnResourceSignals: true
    Type: "AWS::AutoScaling::AutoScalingGroup"
  LaunchConfiguration:
    Properties:
      AssociatePublicIpAddress: true
      IamInstanceProfile: !Ref "InstanceProfile"
      ImageId: !Ref "Ami"
      InstanceType: "t2.micro"
      KeyName: !Ref "KeyPairName"
      SecurityGroups: !Split [ ",", !Join [ ",", [ !Ref "SecurityGroupIds" ] ] ]
      UserData:
        Fn::Base64:
          cfn-init.exe -v -s "AWS::StackName" --region "AWS::Region" 
          cfn-signal.exe -e 0 !Ref "WindowsServerWaitHandle"
    Type: "AWS::AutoScaling::LaunchConfiguration"
  WindowsServerWaitHandle:
    Type: "AWS::CloudFormation::WaitConditionHandle"
  WindowsServerWaitCondition:
    DependsOn: "AutoScalingGroup"
    Properties:
      Handle: !Ref "WindowsServerWaitHandle"
      Timeout: "1800"
      Count: 0
    Type: "AWS::CloudFormation::WaitCondition"

Once the EC2 instance is created I see a few logfiles being generated:

UserdataExecution.log

2017/03/05 05:54:47Z: Userdata execution begins
2017/03/05 05:54:47Z: Zero or more than one <persist> tag was not provided
2017/03/05 05:54:47Z: Unregistering the persist scheduled task
2017/03/05 05:54:50Z: Zero or more than one <runAsLocalSystem> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <script> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <powershell> tag was not provided
2017/03/05 05:54:50Z: Zero or more than one <powershellArguments> tag was not provided
2017/03/05 05:54:50Z: Userdata execution done

WindowsIsReadyToConsole.log

2017/03/03 04:46:27Z: Sending "Windows is Ready" message to console is scheduled successfully
2017/03/05 05:54:27Z: Sending windows is ready message started
2017/03/05 05:54:28Z: Opening COM port handle to write to the console
2017/03/05 05:54:30Z: Serial Port in use. Waiting for Serial Port...
2017/03/05 05:54:48Z: Message: Windows is Ready to use
2017/03/05 05:54:48Z: Sending windows is ready message done
like image 488
Nic Avatar asked Mar 05 '17 05:03

Nic


People also ask

What happens when CloudFormation stack creation fails?

If stack creation fails, go to the CloudFormation Resources list in the AWS Management Console to find the log group. Note that if stack creation fails before any instances are launched, a log group might not be created. By default, AWS deletes CloudWatch log groups if stack creation fails.

How do you debug a CloudFormation failure?

log or /var/log/cfn-init. log , to help you debug the instance launch. You can retrieve the logs by logging in to your instance, but you must disable rollback on failure or else AWS CloudFormation deletes the instance after your stack fails to create. You can also publish the logs to Amazon CloudWatch.

What is the default behavior of a CloudFormation stack if creation fails?

CloudFormation will continue to provision the resources until completion or stop on a different failure. Remediate any issues to continue the deployment process. CloudFormation performs the necessary updates before retrying provisioning actions on resources that couldn't be successfully provisioned earlier.


2 Answers

TLDR

This is a generic error which occurs when the EC2 cannot send a success signal to the ASG. There are many possible reasons why this might occur, but most likely whatever healthcheck it is you use is not working as intended.

Using the below userData should hard-code the healthcheck which is a great way to start testing your application and Cloud Formation template.

My issues

I removed all references to AWS::CloudFormation::WaitConditionHandle and AWS::CloudFormation::WaitCondition

There were issues with my UserData script:

  • The script needed <script> tags to be executed
  • The commands didn't have the correct parameters
  • The variables were not properly injected (for example ${AWS::StackName})

The result is:

UserData:
  "Fn::Base64":
    !Sub |
      <script>
        cfn-init.exe -v --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}
        cfn-signal.exe -e 0 --stack ${AWS::StackName} --resource AutoScalingGroup --region ${AWS::Region}
      </script>
like image 142
Nic Avatar answered Oct 20 '22 15:10

Nic


You are missing the - cloudformation:SignalResource action in your IAM role's PolicyDocument. This permission is required to send signals.

like image 37
spg Avatar answered Oct 20 '22 14:10

spg