Logs for actions on amazon s3 / other AWS services

Question

I am trying to see which user was responsible for changes in S3 (at buckets level). I could not find a audit trail for actions done at S3 bucket level or EC2 who created instances. Beanstalk has a log of the actions the machine performed, but not which user.

Is there a way around AWS that we can see this information in IAM or any other location ?

P.S: I am not interested to know about S3 log buckets which provide access logs

Answer 1

Update

AWS has just announced AWS CloudTrail, finally making auditing API calls available as of today (and for free), see the introductory post AWS CloudTrail - Capture AWS API Activity for details:

Do you have the need to track the API calls for one or more AWS accounts? If so, the new AWS CloudTrail service is for you.

Once enabled, AWS CloudTrail records the calls made to the AWS APIs using the AWS Management Console, the AWS Command Line Interface (CLI), your own applications, and third-party software and publishes the resulting log files to the Amazon S3 bucket of your choice. CloudTrail can also issue a notification to an Amazon SNS topic of your choice each time a file is published. Each call is logged in JSON format for easy parsing and processing.

Please note the following (temporary) constraints:

• Not all services are covered yet, though the most important ones are included in the initial release already and AWS plans to add support for additional services over time.
  • Update: AWS has recently added Seven New Services, and another one today, see below.
• More importantly, not all regions are supported yet (right now the US East (Northern Virginia), and US West (Oregon) Regions only), though AWS will be adding support for additional Regions as quickly as possible.
  • Update: AWS has just added More Locations and Services, quickly approaching coverage of their entire Global Infrastructure indeed.

---

Initial Answer

This is a long standing feature request, but unfortunately AWS does not provide (public) audit trails as of today - the most reasonable way to add this feature would probably be a respective extension to AWS Identity and Access Management (IAM), which is the increasingly ubiquitous authentication and authorization layer for access to AWS resources across all existing (and almost certainly future) Products & Services.

Accordingly there are a few respective answers provided within the IAM FAQs along these lines:

• Will AWS Identity and Access Management administrative actions be logged to an audit trail?:
  No. This is planned for a future release.
• Will user actions in AWS services be logged to an audit trail?
  No. This is planned for a future release.

Answer 2

Current pricing for a single CloudTrail is free.

1. Enable CloudTrail

Use the CloudTrail dashboard and send all events to an S3 bucket, e.g. my-cloudtrail

2. Go Through the Results

The CloudTrail dashboard let's you do some cursory searches, but if you have many thousands of events, it's a pain to use.

Let's say I want actions for user foo_user, I just use the CLI tool:

mkdir -p /tmp/cloudtrail
cd /tmp/cloudtrail
aws s3 sync s3://mc10-cloudtrail .
cd AWSLogs
zcat `find . -type f` | jq '.Records[] | "\(.eventName) \(.userIdentity.userName)"' | grep food_user | sort | uniq

Example Output:

"CreateGrant foo_user"
"DescribeInstances foo_user"
"GetConsoleOutput foo_user"
"ModifyInstanceAttribute foo_user"
"StartInstances foo_user"
"StopInstances foo_user"

Note: S3 data events are billed differently in CloutTrail, but this is somewhat redundant, because you can just enable logging on your S3 bucket and grep those logs, or point them at Logstash/Kibana.