Encrypted Vs Unencrypted EBS Volumes AWS

Question

We are testing standard EBS volume, EBS volume with encryption on EBS optimized m3.xlarge EC2 instance.

While analyzing the test results, we came to know that

EBS volume with encryption is taking lesser time during read, write, read/write operations as compared to EBS without encryption. I think there will be an effect of latency on encrypted EBS volume because of extra encryption overhead on every I/O request.

What will be the appropriate reason why EBS encrypted volumes are faster than normal EBS volumes??

Expected results should be that EBS should yield better results that Encrypted EEBS.

Results :

Encrpted EBS results:

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8
Initializing random number generator from timer.


Extra file open flags: 16384
8 files, 512Mb each
4Gb total file size
Block size 16Kb
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Threads started!
Done.

Operations performed:  0 Read, 262144 Write, 8 Other = 262152 Total
Read 0b  Written 4Gb  Total transferred 4Gb  (11.018Mb/sec)
  705.12 Requests/sec executed

Test execution summary:
    total time:                          371.7713s
    total number of events:              262144
    total time taken by event execution: 2973.6874
    per-request statistics:
         min:                                  1.06ms
         avg:                                 11.34ms
         max:                               3461.45ms
         approx.  95 percentile:               1.72ms

EBS results:

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8
Initializing random number generator from timer.


Extra file open flags: 16384
8 files, 512Mb each
4Gb total file size
Block size 16Kb
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Threads started!
Done.

Operations performed:  0 Read, 262144 Write, 8 Other = 262152 Total
Read 0b  Written 4Gb  Total transferred 4Gb  (6.3501Mb/sec)
  406.41 Requests/sec executed

Test execution summary:
    total time:                          645.0251s
    total number of events:              262144
    total time taken by event execution: 5159.7466
    per-request statistics:
         min:                                  0.88ms
         avg:                                 19.68ms
         max:                               5700.71ms
         approx.  95 percentile:               6.31ms

please help me resolve this issue.

Answer 1

That's certainly unexpected conceptually and also confirmed by Amazon EBS Encryption:

[...] and you can expect the same provisioned IOPS performance on encrypted volumes as you would with unencrypted volumes with a minimal effect on latency. You can access encrypted Amazon EBS volumes the same way you access existing volumes; encryption and decryption are handled transparently and they require no additional action from you, your EC2 instance, or your application. [...] [emphasis mine]

Amazon EBS Volume Performance provides more details on EBS performance in general - from that angle, but pure speculation, maybe the use of encryption implies some default Pre-Warming Amazon EBS Volumes:

When you create any new EBS volume (General Purpose (SSD), Provisioned IOPS (SSD), or Magnetic) or restore a volume from a snapshot, the back-end storage blocks are allocated to you immediately. However, the first time you access a block of storage, it must be either wiped clean (for new volumes) or instantiated from its snapshot (for restored volumes) before you can access the block. This preliminary action takes time and can cause a 5 to 50 percent loss of IOPS for your volume the first time each block is accessed. [...]

Either way, I suggest to rerun the benchmark after pre-warming both new EBS volumes, in case you haven't done so already.