Menu
What are good tests to benchmark a crypto library?
Which unit (time,CPU cycles...) should we use to compare the differents crypto libraries?
Are there any tools, procedures....?
Any Idea, comment is welcome!
Thank you for your inputs!
623
Benchmarking is a topic that arises on occasion on the mailing list. Benchmarking allows you to measure performance and compare the Crypto++ to other libraries like Botan and OpenSSL. The benchmark framework also allows you to gauge the performance of algorithms you add to the library.
The Crypto++ library uses other libraries to ensure the benchmark results of the algorithms are consistent with other libraries. You can use libraries Botan and OpenSSL to cross validate results if you have questions about the Crypto++ results.
A sample program is provided at the end of the article. It benchmarks a block cipher using a ThreadUserTimer but it can be adapted to just about any Crypto++ object. Below is a typical command to run the benchmark program. The first letter, b, means run the benchmarks. The second argument is 2 and it means run each test for about 2 seconds.
The benchmarks are run by name and the name is registered in regtestN.cpp, where N is a number like 1 or 2. The algorithm's name comes from the static member function StaticAlgorithmName. The name is usually a standard cryptographic algorithm name, like SHA2-256 or AES/CBC .
I assume you mean performance benchmarks. I would say that both time and cycles are valid benchmarks, as some code may execute differently on different architectures (perhaps wildly differently if they're different enough).
If it is extremely important to you, I would do the testing myself. You can use some timer (almost all languages have one) or you can use some profiler (almost all languages have one of these too) to figure out the exact performance for the algorithms you are looking for on your target platform.
If you are looking at one algorithm vs. another one, you can look for data that others have already gathered and that will give you a rough idea. For instance, here are some benchmarks from Crypto++: http://www.cryptopp.com/benchmarks.html
Note that they use MB/Second and Cycles/Byte as metrics. I think those are very good choices.
106
Some very good answers before me, but keep in mind optimizations are a very good way to leak key material by timing attack (for example see how devastating it can be for AES). If there is any chance an attacker can time your operations you want not the fastest but the most constant time library available (and possibly the most constant power usage available, if there is any chance someone can monitor yours). OpenSSL does a great job of keeping on top of current attacks, can't necessarily say the same things of other libraries.
26
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
