SSL Java java.io.IOException: Invalid keystore format

Question

I am testing SSL in java with SSLServerSocket and other classes in the java.ssl package. When I run the following code, I get the exception java.io.IOException: Invalid keystore format. My code:

package testing;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.Socket;
import java.security.KeyStore;

import javax.net.ServerSocketFactory;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLServerSocket;
import javax.net.ssl.TrustManager;

public class SSLServerTest {
    public static void main(String[] args) {
        try {
            int port = 3000;
            SSLContext sc = SSLContext.getInstance("TLSv1.2");
            KeyStore ks = KeyStore.getInstance("JKS");
            InputStream ksIs = new FileInputStream("key.txt");
            try {
                ks.load(ksIs, "Bennett556".toCharArray());
            } finally {
                if (ksIs != null) {
                    ksIs.close();
                }
            }
            KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
            kmf.init(ks, "Bennett556".toCharArray());
            sc.init(kmf.getKeyManagers(), new TrustManager[] {}, null);
            ServerSocketFactory ssocketFactory = sc.getServerSocketFactory();
            SSLServerSocket ssocket = (SSLServerSocket) ssocketFactory
                    .createServerSocket(port);
            ssocket.setEnabledProtocols(new String[] { "SSLv3" });
            Socket socket = ssocket.accept();
            BufferedReader in = new BufferedReader(new InputStreamReader(
                    socket.getInputStream()));
            PrintWriter out = new PrintWriter(socket.getOutputStream());
            out.println("Hello, Securly!");
            out.close();
            in.close();
            out.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

File key.txt: 1268312345812304612348712634283427346 I am guessing I should put something else in the key.txt file, but I do not know what to put in it. Probably a searilized object.

EDIT: Client Code:

package testing;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.security.KeyStore;

import javax.net.SocketFactory;
import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.TrustManager;

public class SSLClientTest {
    public static void main(String[] args) {

        int port = 3000;
        String host = "localhost";

        try {
            SSLContext sc = SSLContext.getInstance("TLSv1.2");
            KeyStore ks = KeyStore.getInstance("JKS");
            InputStream ksIs = new FileInputStream("key.txt");
            try {
                ks.load(ksIs, "Bennett556".toCharArray());
            } finally {
                if (ksIs != null) {
                    ksIs.close();
                }
            }
            KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
            kmf.init(ks, "Bennett556".toCharArray());
            sc.init(kmf.getKeyManagers(), new TrustManager[] {}, null);
            SocketFactory factory = sc.getSocketFactory();
            SSLSocket socket = (SSLSocket) factory.createSocket(host, port);
            socket.startHandshake();
            BufferedReader in = new BufferedReader(new InputStreamReader(
                    socket.getInputStream()));
            String str = "";
            while ((str = in.readLine()) != null)
                System.out.println(str);
            in.close();
            socket.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Answer 1

For my case i have used different versions of keytool and java, once i switched to same jdk they got fixed.

Answer 2

I had exactly the same issue. Indeed, the keystore file was invalid and not related to the JDK//JRE version. The problem in my case was caused by Maven. I was using the following option in my pom file:

<resources>
        <resource>
            <directory>src/main/resources</directory>
            <filtering>true</filtering>
        </resource>
    </resources>

The "true" value in the filtering was messing with the key file. Therefore, the keyfile that was available in my classpath when Spring run was not exactly the same I had under my directory "src/main/resources" and that caused the Invalid Keystore Format exception. When I tested with keytool I was using the one under the "resources" folder so that was misleading the real issue.

Solving the issue: in your pom.xml file, change the value for "filtering" to "false". Another way of solving the issue was to specify explicitly the location of the keystore in the application.properties file. So instead of:

server.ssl.key-store: classpath:keystore.jks

I used

server.ssl.key-store: keystore/keystore.jks

Answer 3

Your file is invalid. You have to import a JKS keystore file and not a txt one. You have to use the keytool to create your keystore file and then import this file.