InvalidSignature with python cryptography

Question

I want to verify a signature of some payload, given a public ECDSA key, and I know beforehand that the signature is correct. I want to use the cryptography python library, but the problem being, I can't make the verification work and always get a InvalidSignature exception, even though the signature should be correct.

Here the code snippet I'm currently using. The public key is base64 encoded and in DER format (so no ---BEGIN PUBLIC KEY --- etc.) and the signature is base64 encoded as well. The message is some JSON data as a string, with no spaces.

import base64
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import serialization, hashes
from cryptography.hazmat.backends import default_backend

def cryptography_verify(signature: str, public_key: str, message: str):
    public = base64.b64decode(public_key)
    pub = serialization.load_der_public_key(public, default_backend())

    sig = base64.b64decode(signature)

    msg = bytearray(message, 'utf-8')

    return pub.verify(sig, msg, ec.ECDSA(hashes.SHA256()))

This will result in the following error.

Traceback (most recent call last):
  File "verify.py", line 49, in <module>
    test()
  File "verify.py", line 44, in test
    print(cryptography_verify(signature, public_key, message))
  File "verify.py", line 31, in cryptography_verify
    return pub.verify(sig, msg, ec.ECDSA(hashes.SHA256()))
  File "/home/philipp/.local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/ec.py", line 352, in verify
    _ecdsa_sig_verify(self._backend, self, signature, data)
  File "/home/philipp/.local/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/ec.py", line 101, in _ecdsa_sig_verify
    raise InvalidSignature
cryptography.exceptions.InvalidSignature

The reason I know the signature works for sure, is because I tried out another library called ecdsa, where I can successfully verify the signature. Here the snippet for that.

import hashlib
import base64
import ecdsa

def ecdsa_verify(signature: str, public_key: str, message: str):
    public = base64.b64decode(public_key)
    pub = ecdsa.VerifyingKey.from_der(public)

    sig = base64.b64decode(signature)

    msg = bytearray(message, 'utf-8')

    return pub.verify(sig, msg, hashfunc=hashlib.sha256)

This will just return True. The reason why I'm not just using the working solution is, because I have to use the cryptography library eventually, for some functionality that the ecdsa is not providing. Plus I don't want to use two libraries for the same purpose.

After doing some digging, trying prehashing the message with no positive results, I tried printing out the public key bytes of both deserialized keys (meaning the pub variable).

# for ecdsa library
print(pub.to_string())

# for cryptography library
print(pub.public_bytes(serialization.Encoding.DER, serialization.PublicFormat.SubjectPublicKeyInfo))

Interestingly enough, this resuts in the following.

# for ecdsa library
b'3Le\xf0^g\xc0\x85w \n\xee\xd4\xf7\xfc\xe5`\xa8\xe1\xc7\xd39\x0fu\x8e\x1cUi\r\xf1\x1c\xc7\x96\xe3}*\xed\x1e\x07\xfe\xd2f\x01u\x19\x05\xef\xa795\xfc\xa6\x0bf\xac\xbaS\xf8{\xbf\x1f\xbaT\x87'

# for cryptography library
b'0Y0\x13\x06\x07*\x86H\xce=\x02\x01\x06\x08*\x86H\xce=\x03\x01\x07\x03B\x00\x043Le\xf0^g\xc0\x85w \n\xee\xd4\xf7\xfc\xe5`\xa8\xe1\xc7\xd39\x0fu\x8e\x1cUi\r\xf1\x1c\xc7\x96\xe3}*\xed\x1e\x07\xfe\xd2f\x01u\x19\x05\xef\xa795\xfc\xa6\x0bf\xac\xbaS\xf8{\xbf\x1f\xbaT\x87'

Meaning the cryptography library prepends some bytes to the public key, in comparison to the ecdsa library. Why and how can I prevent this? I feel like I'm just misusing the library and this can somehow be solved, but I just don't know how.

---

Update1: To clarify things more, here the invocations of the verification methods.

def test():
    file_path = "sample.json"
    with open(file_path, "r") as file:
        file_json = json.load(file)

    signature = '9CMVpSkDaKUmZFoluiURVyjJGZ3GgcY1ZopPmw8qR+TsbEH2wbh4zkZDHcNzvV8MeFVn2ln5PuLv2v/+24AMSg=='
    public_key = 'MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEM0xl8F5nwIV3IAru1Pf85WCo4cfTOQ91jhxVaQ3xHMeW430q7R4H/tJmAXUZBe+nOTX8pgtmrLpT+Hu/H7pUhw=='
    message = json.dumps(file_json, separators=(',', ':'))

    print(ecdsa_verify(signature, public_key, message))
    print()
    print(cryptography_verify(signature, public_key, message))
    return

The sample.json looks like this.

{
    "_type": "Targets",
    "delegations": {
        "keys": {},
        "roles": []
    },
    "expires": "2023-01-09T11:31:27.627615676+01:00",
    "targets": {
        "v1": {
            "hashes": {
                "sha256": "E4irx6ElMoNsOoG9sAh0CbFSCPWuunqHrtz9VtY3wUU="
            },
            "length": 1994
        },
        "v2": {
            "hashes": {
                "sha256": "uKOFIodqniVQ1YLOUaHYfr3GxXDl5YXQhWC/1kb3+AQ="
            },
            "length": 1994
        }
    },
    "version": 2
}

As I understand it, both methods use the exact same inputs, so there shouldn't be any difference in the JSON message. I also hex-encoded the deserialized public keys, here you go.

# for ecdsa
334c65f05e67c08577200aeed4f7fce560a8e1c7d3390f758e1c55690df11cc796e37d2aed1e07fed26601751905efa73935fca60b66acba53f87bbf1fba5487

# for cryptography
3059301306072a8648ce3d020106082a8648ce3d03010703420004334c65f05e67c08577200aeed4f7fce560a8e1c7d3390f758e1c55690df11cc796e37d2aed1e07fed26601751905efa73935fca60b66acba53f87bbf1fba5487

Answer 1

The signature format that you provided isn't suitable for OpenSSL. OpenSSL's error can be extracted by augmenting the cryptography method that is called when OpenSSL throws an error:

def _consume_errors(lib):
    errors = []
    while True:
        code = lib.ERR_get_error()
        if code == 0:
            break
        print(hex(code))  # <-- ADD THIS

        err_lib = lib.ERR_GET_LIB(code)
        err_func = lib.ERR_GET_FUNC(code)
        err_reason = lib.ERR_GET_REASON(code)

        errors.append(_OpenSSLError(code, err_lib, err_func, err_reason))
        print('ERROR: ', err_lib, err_func, err_reason)

    return errors

You'll receive error codes d0680a8 and d07803a which can be translated using the OpenSSL CLI:

$ openssl errstr d0680a8    
error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag
$ openssl errstr d07803a    
error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error

You can achieve the same result using the OpenSSL CLI, avoiding the cryptography library altogether. To do so, store the public key in a file:

$ cat pub.key 
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEM0xl8F5nwIV3IAru1Pf85WCo4cfT
OQ91jhxVaQ3xHMeW430q7R4H/tJmAXUZBe+nOTX8pgtmrLpT+Hu/H7pUhw==
-----END PUBLIC KEY-----

Store the signature (base64-decoded) in a file:

echo -n '9CMVpSkDaKUmZFoluiURVyjJGZ3GgcY1ZopPmw8qR+TsbEH2wbh4zkZDHcNzvV8MeFVn2ln5PuLv2v/+24AMSg==' | base64 -d > sig

Verify that we got the correct input, by comparing the digest computed by OpenSSL:

$ echo -ne '{"_type":"Targets","delegations":{"keys":{},"roles":[]},"expires":"2023-01-09T11:31:27.627615676+01:00","targets":{"v1":{"hashes":{"sha256":"E4irx6ElMoNsOoG9sAh0CbFSCPWuunqHrtz9VtY3wUU="},"length":1994},"v2":{"hashes":{"sha256":"uKOFIodqniVQ1YLOUaHYfr3GxXDl5YXQhWC/1kb3+AQ="},"length":1994}},"version":2}' | openssl dgst -sha256
(stdin)= e46bb43c417cac7d72ba24d48a7c5d669afaa88129f5a73ac3c7da1f9a3ae409

And finally try to check the signature:

$ echo -ne '{"_type":"Targets","delegations":{"keys":{},"roles":[]},"expires":"2023-01-09T11:31:27.627615676+01:00","targets":{"v1":{"hashes":{"sha256":"E4irx6ElMoNsOoG9sAh0CbFSCPWuunqHrtz9VtY3wUU="},"length":1994},"v2":{"hashes":{"sha256":"uKOFIodqniVQ1YLOUaHYfr3GxXDl5YXQhWC/1kb3+AQ="},"length":1994}},"version":2}' | openssl dgst -sha256 -verify pub.key -signature sig
Error Verifying Data
140338533991616:error:0D0680A8:asn1 encoding routines:asn1_check_tlen:wrong tag:../crypto/asn1/tasn_dec.c:1130:
140338533991616:error:0D07803A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:290:Type=ECDSA_SIG