Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kinesis Iterator age vs Lambda Iterator Age

What is the difference between the two? Also, what does it mean to have a kinesis iterator age of 12 hours and lambda iterator age of 2 hours. The configuration is very simple & basic (i.e only 1 event source mapping : Kinesis stream -> Lambda)

  • Parallelization factor = 1
  • Number of Shards = 1
  • No enhanced Fan-out.
like image 815
soupybionics Avatar asked Jul 20 '26 07:07

soupybionics


1 Answers

There can be multiple consumers of a single kinesis stream. For example, an EC2 instance, ECS container and a lambda function can read the same stream concurrently.

GetRecords.IteratorAge is a global (per stream) metric. It measures "The age of the last record in all GetRecords calls made against a Kinesis stream", regardless of which consumer called it.

lambda iterator age is only an individual lambda function specific measure. It will not account for other consumers. It is constructed based on MillisBehindLatest value which each GetRecords call returns.

what does it mean to have a kinesis iterator age of 12 hours and lambda iterator age of 2 hours.

This means that the given lambda is 2 hours behind in processing the stream. However, since a stream can be read by multiple consumers, "kinesis iterator age of 12 hours" means that there is some other consumer which is 12 hours behind.

like image 112
Marcin Avatar answered Jul 22 '26 20:07

Marcin