Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ResourceInitializationError: failed to validate logger args: : signal: killed

Tags:

aws-fargate

Suddenly getting the message " ResourceInitializationError: failed to validate logger args: : signal: killed" while starting AWS ECS Fargate Service. Same service was running fine couple of days back.

Following is log driver configurations in related aws task:

Log Configuration
Log driver: awslogs
Key Value
awslogs-group /ecs/analytics
awslogs-region us-east-1
awslogs-stream-prefix ecs

Any idea or help?

like image 255
Amit Kamboj Avatar asked Oct 05 '20 13:10

Amit Kamboj


4 Answers

I finally found the root cause:

The error appears if the fargate service is not able to connect to the CloudWatch api endpoint. This might happen if you have fargate running in a private subnet without internet access. You could either add the CloudWatch log Endpoint to your private subnet or add internet connectivity

like image 177
davidgiga1993 Avatar answered Nov 12 '22 16:11

davidgiga1993


I recently spent hours on this same issue. It turns out that the log group and stream prefix specified in my container definition didn't exist.

It would be wonderful if AWS could provide helpful error messages...

like image 43
Drakee510 Avatar answered Nov 12 '22 15:11

Drakee510


I just experienced this. I have ECS Fargate running and I've just added a VPC endpoint for Cloudwatch Logs com.amazonaws.REGION.logs in my account. When I added the VPC endpoint my logs stopped appearing.

In order to remedy this without deleting the VPC endpoint again, for my setup with Fargate running with internet access I had to ensure that:

  1. My ECS service had a security group rule that to allows HTTPS traffic outbound

    {
       type: egress
       port_to: 443   
       port_from: 443
       protocol: TCP
    }
    
  2. That my new VPC Endpoint had a security group rule to allow HTTPS traffic inbound from my ECS security group

    {
       type: ingress
       port_to: 443   
       port_from: 443
       protocol: TCP
       source_security_group_id: [Your ECS SECURITY GROUP ID]
    }
    
like image 2
Luke Avatar answered Nov 12 '22 16:11

Luke


Came across this issue today. The issue was that the log group I specified didn't exist yet. If you don't want to manually create it, make sure to add the awslogs-create-group and set it to "true". You'll have to grant your ECS Task Execution role a logs:CreateLogGroup permission as well.

  "logConfiguration": {
    "logDriver": "awslogs",
    "secretOptions": null,
    "options": {
      "awslogs-create-group": "true",
      "awslogs-group": "/ecs/app",
      "awslogs-region": "ap-southeast-2",
      "awslogs-stream-prefix": "ecs"
    }
  }

Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_awslogs.html

like image 3
Renzo Sunico Avatar answered Nov 12 '22 16:11

Renzo Sunico