What information is recorded in the azcopy log PERF lines?

Question

As documented in the wiki AzCopy generates PERF lines in its logs to record various elements of the application state.

Here as an extract from an upload job:

2023/08/11 04:49:48 PERF: primary performance constraint is Unknown. States: X:  0, O:  0, M:  0, L:  0, R:  0, D:  1, W: 94, F:  0, B: 63, E:  0, T: 158, GRs: 64
2023/08/11 04:49:50 PERF: primary performance constraint is Unknown. States: X:  0, O:  0, M:  0, L:  0, R:  0, D:  1, W: 55, F:  0, B: 63, E:  0, T: 119, GRs: 64
2023/08/11 04:49:52 PERF: primary performance constraint is Disk. States: X:  0, O:  0, M:  0, L:  0, R:  0, D:  1, W:  7, F:  0, B: 63, E:  0, T: 71, GRs: 64
2023/08/11 04:49:54 PERF: primary performance constraint is Unknown. States: X:  0, O:  0, M:  0, L:  0, R:  0, D:  0, W:  0, F:  0, B: 16, E:  0, T: 16, GRs: 64
2023/08/11 04:49:56 PERF: primary performance constraint is Unknown. States: X:  0, O:  0, M:  0, L:  0, R:  0, D:  0, W:  0, F:  0, B:  0, E:  0, T:  0, GRs: 64

And an extract from a download job:

2023/08/18 09:04:54 PERF: primary performance constraint is Unknown. States: C:  0, R: 64, W: 1507, F:  0, H:  0, B: 16, S:  0, P:  8, Q:  0, D:  0, E:  0, T: 1595, GRs: 16
2023/08/18 09:04:56 PERF: primary performance constraint is Unknown. States: C:  0, R: 64, W: 1513, F:  0, H:  1, B: 15, S:  0, P:  4, Q:  0, D:  0, E:  0, T: 1597, GRs: 16
2023/08/18 09:04:58 PERF: primary performance constraint is Unknown. States: C:  0, R: 64, W: 1514, F:  0, H:  1, B: 15, S:  0, P:  1, Q:  0, D:  0, E:  0, T: 1595, GRs: 16
2023/08/18 09:05:00 PERF: primary performance constraint is Unknown. States: C:  0, R: 64, W: 1512, F:  0, H:  1, B: 15, S:  0, P:  4, Q:  0, D:  0, E:  0, T: 1596, GRs: 16
2023/08/21 02:01:15 PERF: primary performance constraint is Unknown. States: C:  0, R:  0, W:  0, F:  0, H:  0, B:  0, S:  0, P:  0, Q:  0, D:  0, E:  0, T:  0, GRs: 16

There is clearly a large amount of data being recorded here but unfortunately as far as I can tell the wiki doesn't document what each of these fields represent.

Is this information known and available anywhere?

Answer 1

The file that you have shared can be breakdown as given below:

I have taken reference from one of the line also stated below:

2023/08/18 09:04:54 PERF: primary performance constraint is Unknown. States: C:  0, R: 64, W: 1507, F:  0, H:  0, B: 16, S:  0, P:  8, Q:  0, D:  0, E:  0, T: 1595, GRs: 16

• Date and Time: 2023/08/18 09:04:54 - This is the date and time when the log entry was generated.

• Message Type: PERF - This indicates that the log entry is related to performance metrics.

• Primary Performance Constraint: Unknown - In this context, it states that the primary performance constraint is unknown. This could mean that azcopy is not able to determine the primary limiting factor affecting performance.

• States:

  • C: 0 - Number of Concurrency (parallel operations) currently active.
  • R: 64 - Number of HTTP Requests (usually indicating read operations).
  • W: 1507 - Number of Write (upload) operations.
  • F: 0 - Number of Failed operations.
  • H: 0 - Number of OnHold operations.
  • B: 16 - Number of Buffers (part of internal caching mechanisms).
  • S: 0 - Number of Scheduled (pending) operations.
  • P: 8 - Number of Pipelines (concurrent transfer channels).
  • Q: 0 - Number of Queued operations.
  • D: 0 - Number of Dispatched (started) operations.
  • E: 0 - Number of Executing operations.
  • T: 1595 - Total number of operations performed.
  • GRs: 16 - Number of Get Responses (usually indicating read operations).