Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Firehose is unable to assume role

I'm trying to use Firehose API (JS) and I keep getting the following error:

"InvalidArgumentException: Firehose is unable to assume role arn:aws:iam::XXXXXXXXXX:role/NAME. Please check the role provided.

I check the role and I have set my custom policy to include all resources for STS and Firehose action. I have no clue why this error exists if I'm allowing AssumeRole under STS.

Method Calling

The method I'm using is createDeliveryStream(params = {}, callback) with S3DestinationConfiguration.

Policy JSON

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "iam:*",
                "kinesisvideo:*",
                "s3:PutAccountPublicAccessBlock",
                "s3:GetAccountPublicAccessBlock",
                "s3:ListAllMyBuckets",
                "s3:*",
                "firehose:*",
                "sts:*",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        }
    ]
}
like image 305
Brian Nezhad Avatar asked Apr 25 '19 17:04

Brian Nezhad


People also ask

Can a firehose assume a role?

Kinesis Data Firehose assumes that IAM role and gains access to the specified bucket, key, and CloudWatch log group and streams. Use the following access policy to enable Kinesis Data Firehose to access your S3 bucket and AWS KMS key. If you don't own the S3 bucket, add s3:PutObjectAcl to the list of Amazon S3 actions.

Is firehose fully managed?

Amazon KinesisAmazon KinesisCapture,process,andstoredatastreams Amazon Kinesis Data Streams is a scalable and durable real-time data streaming service that can continuously capture gigabytes of data per second from hundreds of thousands of sources.https://aws.amazon.com › kinesisAmazon Kinesis - Process & Analyze Streaming Data - AWS Data Firehose is a fully managed service for delivering real-time streaming datastreaming dataStreaming data is data that is generated continuously by thousands of data sources, which typically send in the data records simultaneously, and in small sizes (order of Kilobytes).https://aws.amazon.com › streaming-dataWhat Is Streaming Data? | Amazon Web Services (AWS) to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service, Splunk, and any custom HTTP endpoint or HTTP endpoints owned by supported third-party service providers, ...

What is buffer size in Kinesis firehose?

Kinesis Data Firehose buffers incoming data before delivering it to Amazon S3. You can configure the values for S3 buffer size (1 MB to 128 MB) or buffer interval (60 to 900 seconds), and the condition satisfied first triggers data delivery to Amazon S3.

Can firehose convert to parquet?

Amazon Kinesis Data Firehose can convert the format of your input data from JSON to Apache Parquet or Apache ORC before storing the data in Amazon S3.


1 Answers

Please check the trust relationship defined in the IAM role and verify that it is set to firehose.amazonaws.com. Policy defined in Trust relationship enables services to assume the role. For Kinesis Firehose, refer this document which contains details about IAM roles for Firehose. Refer "Grant Kinesis Data Firehose Access to an Amazon S3 Destination" section in the document which mentions about trust policy.

like image 72
krishna_mee2004 Avatar answered Sep 23 '22 09:09

krishna_mee2004