Why does Java's X22519 Lib works with some inputs but not others

Question

I am using java built-in SunEC 21 security provider to do X25519 test cases and all work but one.
I am using the same logic for every test just different inputs.
When I use the inputs:

Public key: e5210f12786811d3f4b7959d0538ae2c31dbe7106fc03c3efc4cd549c715a493
Private key: 4b66e9d4d1b4673c5ad22691957d6af5c11b6421e0ea01d42ca4169e7918ba0d

the expected output is:

95cbde9476e8907d7aade45cb4b873f88b595a68799fa152e6f8f7647aac7957

Instead I get:

d5f33573c9f6b8129483acce1e2534e95d3c41af6b00d0d30437b87cada57e4a

(This is test case two in the RFC 7748.)

I don't know why this is. Whenever I try it with the first or third input in the RFC 7748 it works fine. (Again, same logic just different inputs.)

byte[] publicKey;
publicKey = Hex.decode("e5210f12786811d3f4b7959d0538ae2c31dbe7106fc03c3efc4cd549c715a493");

byte[] privateKey;// input scalar
privateKey = Hex.decode("4b66e9d4d1b4673c5ad22691957d6af5c11b6421e0ea01d42ca4169e7918ba0d");
// System.out.println("bc go crazy:" + Hex.toHexString(privateKey));

NamedParameterSpec paramSpec = new NamedParameterSpec("X25519");

BigInteger clientBigInteger = new BigInteger(1, Util.reverse(publicKey));

KeyFactory kf = KeyFactory.getInstance("X25519");
XECPublicKeySpec clientPublicKeySpec = new XECPublicKeySpec(paramSpec, clientBigInteger);

XECPublicKey clientPublicKey;

clientPublicKey = (XECPublicKey) kf.generatePublic(clientPublicKeySpec);

XECPrivateKeySpec privateKeySpec = new XECPrivateKeySpec(paramSpec, privateKey);

XECPrivateKey secretKey = (XECPrivateKey) kf.generatePrivate(privateKeySpec);

KeyAgreement ka = KeyAgreement.getInstance("X25519");

ka.init(secretKey);
ka.doPhase(clientPublicKey, true);
byte[] sharedSecret = ka.generateSecret();

System.err.println("---------- X25519 test cases ----------");

System.out.println("X25519-test-case-2-out:" + Hex.toHexString(sharedSecret));
System.out.println(
"X25519-test-case-2-exp:95cbde9476e8907d7aade45cb4b873f88b595a68799fa152e6f8f7647aac7957");

This is the code that's not giving the right output. I have checked test vectors ten times, and copy pasted logic twenty. What's wrong with my code? Why does it work for some test vectors but not others?

Based off RFC 7748.

Answer 1

There is an errata for RFC 7748 that includes, among other issues, a mistake in the u-coordinate of this test vector:

Section 5.2 says:

Input u-coordinate:
e5210f12786811d3f4b7959d0538ae2c31dbe7106fc03c3efc4cd549c715a493

It should say:

Input u-coordinate:
e5210f12786811d3f4b7959d0538ae2c31dbe7106fc03c3efc4cd549c715a413

Notes:

In the X25519 2nd test vector the last byte of input u-coordinate should be 13 instead of 93. This will fix inconsistency between u-coordinate, its base10 representation and the output u-coordinate.

With the changed u-coordinate, your code returns the right result.

---

Edit: Upon closer inspection of the erratum, I saw that the verifier rejected it, meaning that the original u-coordinate should actually work as well:

--VERIFIER NOTES--
A change of one bit of the input u-coordinate in the hexadecimal representation is proposed (to make it "consistent" with the base 10 representation). However, implementations of x25519 should "mask" that bit after taking a u-coordinate as an input - therefore, the existing text of RFC does not have any errors there.<

So why does the Java code not produce the correct result for the original u-coordinate, but only for the changed one?
Both u-coordinates differ only in the most significant bit. For the changed u-coordinate, the most significant bit is 0: 0x13 = 0001 0011, for the original u-coordinate it is 1: 0x93 = 1001 0011 (note that X25519 uses little endian, so the most significant byte is at the end).
Since the X25519 specification stipulates that the most significant bit of the u-coordinate must be cleared and that implementations must ensure this (RFC 7748, sec. 5), both u-coordinates should indeed produce the correct result.
Apparently, contrary to the X25519 specification, the JCA/JCE classes do not implicitly ensure that the most significant bit is set to 0 (which may be a bug), so this must be done explicitly in the Java code or, equivalently, the changed u-coordinate must be used from the outset.

Alternatively, BouncyCastle can be used, which avoids this problem:

import org.bouncycastle.crypto.agreement.X25519Agreement;
import org.bouncycastle.crypto.params.X25519PrivateKeyParameters;
import org.bouncycastle.crypto.params.X25519PublicKeyParameters;
import org.bouncycastle.util.encoders.Hex;

...

byte[] privateKey = Hex.decode("4b66e9d4d1b4673c5ad22691957d6af5c11b6421e0ea01d42ca4169e7918ba0d");
X25519PrivateKeyParameters privateKeyParams = new X25519PrivateKeyParameters(privateKey); 
byte[] publicKey = Hex.decode("e5210f12786811d3f4b7959d0538ae2c31dbe7106fc03c3efc4cd549c715a493");
X25519PublicKeyParameters publicKeyParams = new X25519PublicKeyParameters(publicKey); 

X25519Agreement agreement = new X25519Agreement();
agreement.init(privateKeyParams);
byte[] sharedSecret = new byte[agreement.getAgreementSize()];
agreement.calculateAgreement(publicKeyParams, sharedSecret, 0);
System.out.println(Hex.toHexString(sharedSecret)); // 95cbde9476e8907d7aade45cb4b873f88b595a68799fa152e6f8f7647aac7957

Answer 2

I am sure X25519 public inputs are 32-byte little-endian field elements where the top bit is ignored. if you can supply a spec that takes the raw 32-byte public value (little-endian) and lets the provider decode (mask) it, that might also work.