Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

"An error occurred: LogGroup - <resource name> already exists" while trying to deploy Serverless

after running sls deploy -v && sls s3deploy as I normally do, I ran into this issue:

...

CloudFormation - CREATE_FAILED - AWS::Logs::LogGroup - CallTextractLogGroup

...

CloudFormation - CREATE_FAILED - AWS::IAM::Role - IamRoleStateMachineExecution
CloudFormation - CREATE_FAILED - AWS::Logs::LogGroup - StartTextractStateMachineLogGroup
CloudFormation - CREATE_FAILED - AWS::Lambda::LayerVersion - Boto3LayerLambdaLayer

...

  Serverless Error ---------------------------------------

  An error occurred: StartTextractStateMachineLogGroup - /aws/lambda/textract-service-dev-startTextractStateMachine already exists.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     OS:                     linux
     Node Version:           12.2.0
     Serverless Version:     1.42.3

I don't remember changing anything at all before deployment, other than adding Retry sections to my Step Function States, which shouldn't have messed with CF logs at all.

I tried running sls remove and then my deployment command again, no luck there.

Also tried adding cfLogs: true to my provider section in the YAML, no love!

I tried manually deleting the LogGroup in the CloudWatch console, but it's not even there.

Here is my serverless.yml:

service: textract-service

provider:
  name: aws
  runtime: python3.7
  timeout: 10
  region: us-east-1
  cfLogs: true
  environment:
    STATE_MACHINE_ARN: ${self:resources.Outputs.TextractStepFunctions.Value}
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:PutObject"
        - "states:*"
      Resource:
        Fn::Join:
          - ""
          - - "the-chumiest-bucket-ARN or *"
            - ${self:resources.Outputs.TextractStepFunctions.Value}

plugins:
  - serverless-plugin-existing-s3
  - serverless-step-functions
  - serverless-pseudo-parameters
  - serverless-plugin-existing-s3 
layers:
  boto3Layer:
    package:
      artifact: boto3_layer.zip
    allowedAccounts:
      - "*"

functions:
  startTextractStateMachine:
    handler: src/start_textract_state_machine.lambda_handler
    role: the-chumiest-bucket-role
    layers:
      - {Ref: Boto3LayerLambdaLayer}
    events:
      - existingS3:
          bucket: the-chumiest-bucket
          events:
            - s3:ObjectCreated:*
          rules:
            - prefix: input1/
            - suffix: .pdf
  callTextract:
    handler: src/call_textract.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  getTextractOutput:
    handler: src/get_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}
  parseTextractOutput:
    handler: src/parse_textract_output.lambda_handler
    role: the-chumiest-bucket-role

    layers:
      - {Ref: Boto3LayerLambdaLayer}

stepFunctions:
  stateMachines:
    textractStepFunc:
      name: TextractStepFunctions
      definition:
        Comment: A state machine for the Textract OCR process.
        StartAt: StartTextractStateMachine
        States:
          StartTextractStateMachine:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-startTextractStateMachine"
            Next: CallTextract
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          CallTextract:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-callTextract"
            Next: GetTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          GetTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-getTextractOutput"
            Next: ParseTextractOutput
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
          ParseTextractOutput:
            Type: Task
            Resource: "arn:aws:lambda:#{AWS::Region}:#{AWS::AccountId}:function:#{AWS::StackName}-parseTextractOutput"
            Retry:
              - ErrorEquals:
                - HandledError
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
              - ErrorEquals:
                - States.ALL
                IntervalSeconds: 1
                MaxAttempts: 1
                BackoffRate: 2
            End: true

resources:
  Outputs:
    TextractStepFunctions:
      Description: The ARN of the state machine
      Value:
        Ref: TextractStepFunctions
like image 762
ChumiestBucket Avatar asked May 14 '19 19:05

ChumiestBucket


People also ask

Do we attach to eventually already existing resources on deploy?

In its context we attach to eventually already existing resources on deploy, as it's being requested here. Sorry, something went wrong. I didn't understand the logic of not working! Right now I have critical data in production, does that mean I have to do this externally? So all the logic of creating new environments in a simple way goes to waste?

Does serverless fail if they already exist?

To have it as a hard requirement that serverless fails if they already exist does not make sense. We get around this by using Ansible to deploy these resources, but the idea that Serverless can't handle this scenario makes no sense.

What happens when a resource is not updated in serverless?

If there's a change in a resource, that single resource will be updated (see my previous comment to @bwship ). If no resources are updated, Serverless will generally catch that there was no change (if the CloudFormation template it generates is exactly the same as the last one it deployed).

Why is my API gateway resource not connecting to any resource?

This error is caused when creating a API Gateway resource with a method (GET, PUT, POST, etc.) but not connecting that method to any other resource. In the Stackery visual editor, be sure to link your API routes to functions. Your browser does not support the video tag.


Video Answer


1 Answers

It looks like you have a log group from a previous (failed?) deployment that still exists in CloudWatch Logs.

You should see this log group in the CloudWatch console (not CloudFormation).

You can run this command (AWS CLI):

aws logs delete-log-group --log-group-name /aws/lambda/textract-service-dev-startTextractStateMachine

to delete your log group and then retry to deploy.

like image 88
jogold Avatar answered Oct 22 '22 14:10

jogold