Azure App Insights Sampling (ItemCount)

Question

I have a question about Azure App Insights Sampling. If itemCount field is greater than 1 for a log item, does it mean that there was an exactly the SAME request and it was sampled?

My logs have one request that sends this message with itemCount = 2. And this request has ended with OptimisticConcurrencyException, so my transaction has been roll-backed. In this transaction I send a message to 3rd party service. The most interesting is that they told me they've got 2 messages from my service and my database has been updated (so transaction has been committed). All of it became clear, if there were 2 requests and one of them returned 200 code, and another returned 500. But app insights log item abot OptimisticConcurrencyException has value itemCount = 2, which means that this exception was thrown twice (for both requests). Furthermore Beside this I don't see any other requests, that could change data, that request was changing.

So could anybody explain me how app insights samples requests and errors?

Answer 1

This really depends on how/where your sampling occurred, as sampling could have occurred at 3 different places depending on how you have your app configured.

There's a fair amount of documentation about the various layers of sampling, but hypothetically:

The sampling algorithm decides which telemetry items to drop, and which ones to keep (whether it's in the SDK or in the Application Insights service). The sampling decision is based on several rules that aim to preserve all interrelated data points intact, maintaining a diagnostic experience in Application Insights that is actionable and reliable even with a reduced data set. For example, if for a failed request your app sends additional telemetry items (such as exception and traces logged from this request), sampling will not split this request and other telemetry. It either keeps or drops them all together. As a result, when you look at the request details in Application Insights, you can always see the request along with its associated telemetry items.

Update: I got some more details from people on the team that do the sampling, and it works like this:

1. Sampling ratio is determined by the number of events per second occurring in the app
2. The AI SDK randomly selects requests to be sampled when the request begins (so, it is not known whether it will fail or succeed)
3. AI SDK assigns itemCount=<sampling ratio>

This would then explain the behavior you are seeing, when two requests (success + failure) were counted as two failures: the failed request was sampled "in", and so in telemetry, you'd have 2 failed requests (one request with itemCount=2) instead of a failed and a successful, because the successful one got sampled away.