Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does AWS CloudWatch Events Rule supports any wildcards in S3 bucket/key names

I am trying to create an event rule that is triggered by a change in a file in S3 bucket in different AWS account. Detail description is here

So far the rule works fine with exact file names, but I need to make it work with filename prefixes. In the working example, the file name is an exact string in the non-working example the file name is a wildcard. Does CloudWatch Events Rule JSON pattern supports wildcards?

Working configuration:

{
  "source": ["aws.s3"],
  "account": ["1111111xxxxx"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject"],
    "requestParameters": { "bucketName": ["mybucket"], "key": ["myfile-20180301.csv"] }
  }
}

Non-working configuration:

{
  "source": ["aws.s3"],
  "account": ["1111111xxxxx"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject"],
    "requestParameters": { "bucketName": ["mybucket"], "key": ["myfile-*"] }
  }
}
like image 448
k1r0 Avatar asked Mar 22 '18 18:03

k1r0


People also ask

Can CloudWatch read from S3?

There are several ways that you can use CloudWatch with Amazon S3. Monitor bucket storage using CloudWatch, which collects and processes storage data from Amazon S3 into readable, daily metrics. These storage metrics for Amazon S3 are reported once per day and are provided to all customers at no additional cost.

What are the various way you can control access to the data stored in S3?

You can restrict access even if the users are granted access in an IAM policy. Using Amazon S3 Block Public Access as a centralized way to limit public access. Block Public Access settings override bucket policies and object permissions.

What is bucket key in S3?

Amazon S3 uses this bucket key to create unique data keys for objects in a bucket, avoiding the need for additional KMS requests to complete encryption operations, and this translates to reduction of request traffic from Amazon S3 to KMS, allowing you to access encrypted objects within your S3 buckets at a fraction of ...


2 Answers

I found a fancy solution for this using Content-based filtering (released in February 2020) like prefix for example.

So in your case, the solution should be:

{
  "source": ["aws.s3"],
  "account": ["1111111xxxxx"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject"],
    "requestParameters": {
      "bucketName": ["mybucket"],
      "key": [{ "prefix": "myfile-" }]
    }
  }
}
like image 88
Marto Avatar answered Oct 04 '22 00:10

Marto


The template code gave by Marto was not working for me, however the doc led to a solution:

{
  "source": ["aws.s3"],
  "account": ["1111111xxxxx"],
  "detail": {
    "eventSource": ["s3.amazonaws.com"],
    "eventName": ["PutObject"],
    "requestParameters": {
      "bucketName": ["mybucket"],
      "key": [{"prefix": "myfile-*"}]
    }
  }
}

Hope it helps.

like image 28
dijkstraman Avatar answered Oct 04 '22 01:10

dijkstraman