Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

AWS S3 bucket logs vs AWS cloudtrail

What's the difference between the AWS S3 logs and the AWS CloudTrail? On the doc of CloudTrail I saw this:

CloudTrail adds another dimension to the monitoring capabilities already offered by AWS. It does not change or replace logging features you might already be using.

like image 674
Steve Ritz Avatar asked Dec 07 '15 15:12

Steve Ritz


People also ask

How do I send CloudTrail logs to S3 bucket?

To add the required CloudTrail policy to an Amazon S3 bucketOpen the Amazon S3 console at https://console.aws.amazon.com/s3/ . Choose the bucket where you want CloudTrail to deliver your log files, and then choose Properties. Choose Permissions.

What are the reasons for streaming the AWS CloudTrail logs to AWS S3 bucket?

You can use AWS CloudTrail logs together with server access logs for Amazon S3. CloudTrail logs provide you with detailed API tracking for Amazon S3 bucket-level and object-level operations. Server access logs for Amazon S3 provide you visibility into object-level operations on your data in Amazon S3.

How do I check logs on S3 bucket?

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that you want to enable server access logging for. Choose Properties. In the Server access logging section, choose Edit.

How long will CloudTrail retain event history not S3?

Amazon CloudTrail will only show the results of the CloudTrail Event History for the current region you are viewing for the last 90 days and support the Amazon Web Services services found here. These events are limited to Management Events with create, modify, and delete API calls and account activity.


2 Answers

CloudTrail tracks API access for infrastructure-changing events, in S3 this means creating, deleting, and modifying bucket (S3 CloudTrail docs). It is very focused on API methods that modify buckets.

S3 Server Access Logging provides web server-style logging of access to the objects in an S3 bucket. This logging is granular to the object, includes read-only operations, and includes non-API access like static web site browsing.

like image 176
James Avatar answered Sep 21 '22 23:09

James


AWS has added one more functionality since this question was asked, namely CloudTrail Data events

Currently there are 3 features available:

  1. CloudTrail: Which logs almost all API calls at Bucket level Ref
  2. CloudTrail Data Events: Which logs almost all API calls at Object level Ref
  3. S3 server access logs: Which logs almost all (best effort server logs delivery) access calls to S3 objects. Ref

Now, 2 and 3 seem similar functionalities but they have some differences which may prompt users to use one or the other or both(in our case)! Below are the differences which I could find:

  • Both works at different levels of granularity. e.g. CloudTrail data events can be set for all the S3 buckets for the AWS account or just for some folder in S3 bucket. Whereas, S3 server access logs would be set at individual bucket level
  • The S3 server access logs seem to give more comprehensive information about the logs like BucketOwner, HTTPStatus, ErrorCode, etc. Full list

Information which is not available in Cloudtrail logs but is available in Server Access logs. Reference:

  • Fields for Object Size, Total Time, Turn-Around Time, and HTTP Referer for log records
  • Life cycle transitions, expiration, restores
  • Logging of keys in a batch delete operation
  • Authentication failures
  • CloudTrail does not deliver logs for requests that fail authentication (in which the provided credentials are not valid). However, it does include logs for requests in which authorization fails (AccessDenied) and requests that are made by anonymous users.
  • If a request is made by a different AWS Account, you will see the CloudTrail log in your account only if the bucket owner owns or has full access to the object in the request. If that is not the case, the logs will only be seen in the requester account. The logs for the same request will however be delivered in the server access logs of your account without any additional requirements.

AWS Support recommends that decisions can be made using CloudTrail logs and if you need that additional information too which is not available in CloudTrail logs, you can then use Server access logs.

like image 36
rahuljain1311 Avatar answered Sep 21 '22 23:09

rahuljain1311