Java 1.7 + JSCH: java.security.InvalidKeyException: Key is too long for this algorithm

Question

I'm trying to use JSCH to upload a file to a remote SFTP share. Every time I attempt to connect to the share from within my code, I get an exception that looks something like this:

com.jcraft.jsch.JSchException: Session.connect: java.security.InvalidKeyException: Key is too long for this algorithm
    at com.jcraft.jsch.Session.connect(Session.java:558) ~[jsch-0.1.51.jar:na]
    at com.jcraft.jsch.Session.connect(Session.java:183) ~[jsch-0.1.51.jar:na]

I've seen posts that describe this error when upgrading to Java 8, but we're still on Java 7, and I don't know enough about Java's cryptography support to know if that matters.

Some people suggest installing JCE (Java Cryptography Extensions) to solve this problem, so I gave it a shot, but I still get the same error after copying the appropriate jar files into the /libs/security directory and restarting the application. We confirmed that JCE was installed by executing this script and noting that the exception was not thrown.

I also tried connecting to the remote SFTP share from the terminal using the sftp command in verbose mode. Here's what I got:

OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to XXXXXXXXXXXXX [XXXXXXXXXXXX] port XX.
debug1: Connection established.
debug3: Incorrect RSA1 identifier
debug3: Could not load "/Users/XXXXX/.ssh/id_rsa" as a RSA1 public key
debug1: identity file /Users/XXXXX/.ssh/id_rsa type 1
debug1: identity file /Users/XXXXX/.ssh/id_rsa-cert type -1
debug1: identity file /Users/XXXXX/.ssh/id_dsa type -1
debug1: identity file /Users/XXXXX/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version 3.2.9 SSH Secure Shell
debug1: no match: 3.2.9 SSH Secure Shell
debug2: fd 3 setting O_NONBLOCK
debug3: load_hostkeys: loading entries for host "XXXXXXXXXXXXX" from file "/Users/XXXXX/.ssh/known_hosts"
debug3: load_hostkeys: loaded 0 keys
debug1: SSH2_MSG_KEXINIT sent
debug3: Received SSH2_MSG_IGNORE
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: [email protected],[email protected],[email protected],[email protected],ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,[email protected],[email protected],aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,[email protected],[email protected],aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected]
debug2: kex_parse_kexinit: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-md5,hmac-sha1,[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-md5,hmac-sha1,[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,[email protected],zlib
debug2: kex_parse_kexinit: none,[email protected],zlib
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: kex_parse_kexinit: diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-dss
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,twofish128-cbc,cast128-cbc,twofish-cbc,blowfish-cbc,aes192-cbc,aes256-cbc,twofish192-cbc,twofish256-cbc,arcfour
debug2: kex_parse_kexinit: aes128-cbc,3des-cbc,twofish128-cbc,cast128-cbc,twofish-cbc,blowfish-cbc,aes192-cbc,aes256-cbc,twofish192-cbc,twofish256-cbc,arcfour
debug2: kex_parse_kexinit: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
debug2: kex_parse_kexinit: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: none,zlib
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: 
debug2: kex_parse_kexinit: first_kex_follows 0 
debug2: kex_parse_kexinit: reserved 0 
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-cbc hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-cbc hmac-md5 none
debug2: dh_gen_key: priv key bits set: 122/256
debug2: bits set: 496/1024
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug3: Received SSH2_MSG_IGNORE
debug1: Server host key: DSA XXXXXXXXXXXXXXXXXXXXXXXX
debug3: load_hostkeys: loading entries for host "XXXXXXXXXXXXX" from file "/Users/XXXXX/.ssh/known_hosts"
debug3: load_hostkeys: loaded 0 keys
debug3: load_hostkeys: loading entries for host "XXXXXXXXXXXX" from file "/Users/XXXXX/.ssh/known_hosts"
debug3: load_hostkeys: loaded 0 keys
    The authenticity of host 'XXXXXXXXXXXXX (XXXXXXXXXXXX)' can't be established.
    DSA key fingerprint is XXXXXXXXXXXXXXXXXXXXXXXX.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'XXXXXXXXXXXXX,XXXXXXXXXXXX' (DSA) to the list of known hosts.
debug2: bits set: 516/1024
debug1: ssh_dss_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: Received SSH2_MSG_IGNORE
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug3: Received SSH2_MSG_IGNORE
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/XXXXX/.ssh/id_rsa (0x7f8e28500a10),
debug2: key: /Users/XXXXX/.ssh/id_dsa (0x0),
debug3: Received SSH2_MSG_IGNORE
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/XXXXX/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Received SSH2_MSG_IGNORE
debug1: Authentications that can continue: password
debug3: start over, passed a different list password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup password
debug3: remaining preferred: ,keyboard-interactive,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password

If I'm reading the output correctly (and I may not be) the handshake process settled on using aes128-cbc for key exchange and hmac-md5 for the actual session encryption. According to the JSCH documentation (minimal though it may be), both of these algorithms are supported.

I can connect to this share with both the sftp command-line utility and with FileZilla, so the problem has to either be with JSCH or with my Java configuration, but I'm at a loss to figure out what's what.

Java version:

java version "1.7.0_71"
OpenJDK Runtime Environment
OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode)

JSCH version:

<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.51</version>
</dependency>

Thanks in advance!

EDIT: It looks like a bug for this exact behaviour was filed against the JDK, but was closed with no resolution. There's also an email thread between the maintainers of JSCH and the JDK developers that discusses the issue, but has no resolution.

Answer 1

We ended up swapping JSCH out for SSHJ. It depends on the BouncyCastle crypto libraries rather than on Java's built-in crypto packages, and is capable of connecting to our server with no problems.

Answer 2

You can force JSCH to use SHA256 instead of SHA1 with keysize > 1024 (which JSSE doesn't allow anymore) like this:

java.util.Properties configuration = new java.util.Properties();
configuration.put("kex", "diffie-hellman-group-exchange-sha256");           
configuration.put("StrictHostKeyChecking", "no");
session.setConfig(configuration);