QCryptographicHash - what is SHA3 here in reality?

Question

I got such a piece of code:

void SHAPresenter::hashData(QString data)
{
    QCryptographicHash* newHash = new QCryptographicHash(QCryptographicHash::Sha3_224);
    newHash->addData(data.toUtf8());
    QByteArray hashResultByteArray = newHash->result();
    setHashedData(QString(hashResultByteArray.toHex()));
    delete newHash;
}

According to Qt spec, QCryptographicHash::Sha3_224 should "generate an SHA3-224 hash sum. Introduced in Qt 5.1". I wanted to compare result of that code to something other source to check whether I put data in correct manner. I found site: https://emn178.github.io/online-tools/sha3_224.html So we have SHA3_224 in both cases. The problem is that the first will generate such a byte string from "test":

3be30a9ff64f34a5861116c5198987ad780165f8366e67aff4760b5e

And the second:

3797bf0afbbfca4a7bbba7602a2b552746876517a7f9b7ce2db0ae7b

Not similar at all. But there is also a site that do "Keccak-224": https://emn178.github.io/online-tools/keccak_224.html

And here result is:

3be30a9ff64f34a5861116c5198987ad780165f8366e67aff4760b5e

I know that SHA3 is based on Keccak's functions - but what is the issue here? Which of these two implementations follows NIST FIPS 202 in proper manner and how do we know that?

Answer 1

I'm writing a Keccak library for Java at the moment, so I had the toys handy to test an initial suspicion.

First a brief summary. Keccak is a sponge function which can take a number of parameters (bitrate, capacity, domain suffix, and output length). SHA-3 is simply a subset of Keccak where these values have been chosen and standardised by NIST (in FIPS PUB 202).

In the case of SHA3-224, the parameters are as follows:

bitrate: 1152
capacity: 448
domain suffix: "01"
output length: 224 (hence the name SHA3-224)

The important thing to note is that the domain suffix is a bitstring which gets appended after the input message and before the padding. The domain suffix is an optional way to differentiate different applications of the Keccak function (such as SHA3, SHAKE, RawSHAKE, etc). All SHA3 functions use "01" as a domain suffix.

Based on the documentation, I get the impression that Keccak initially had no domain suffix concept, and the known-answer tests provided by the Keccak team require that no domain suffix is used.

So, to your problem. If we take the String "test" and convert it to a byte array using ASCII or UTF-8 encoding (because Keccak works on binary, so text must be converted into bytes or bits first, and it's therefore important to decide on which character encoding to use) then feed it to a true SHA3-224 hash function we'll get the following result (represented in hexadecimal, 16 bytes to a line for easy reading):

37 97 BF 0A FB BF CA 4A 7B BB A7 60 2A 2B 55 27
46 87 65 17 A7 F9 B7 CE 2D B0 AE 7B

SHA3-224 can be summarised as Keccak[1152, 448](M || "01", 224) where the M || "01" means "append 01 after the input message and before multi-rate padding".

However, without a domain suffix we get Keccak[1152, 448](M, 224) where the lonesome M means that no suffix bits are appended, and the multi-rate padding will begin immediately after the input message. If we feed your same input "test" message to this Keccak function which does not use a domain suffix then we get the following result (again in hex):

3B E3 0A 9F F6 4F 34 A5 86 11 16 C5 19 89 87 AD
78 01 65 F8 36 6E 67 AF F4 76 0B 5E

So this result indicates that the function is not SHA3-224.

Which all means that the difference in output you are seeing is explained entirely by the presence or absence of a domain suffix of "01" (which was my immediate suspicion on reading your question). Anything which claims to be SHA3 must use a "01" domain suffix, so be very wary of tools which behave differently. Check the documentation carefully to make sure that they don't require you to specify the desired domain suffix when creating/using the object or function, but anything which claims to be SHA3 really should not make it possible to forget the suffix bits.

Answer 2

This is a bug in Qt and reported here and Fixed in Qt5.9