Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Spring: Cannot connect to a JMX Server using RMI from behind a firewall

My Spring application is running on a machine that is behind a NAT firewall (pfSense). The machine's internal IP is a.b.c.d, and the NAT IP is w.x.y.z

The Spring configuration's serviceUrl is set to my internal IP (a.b.c.d) on port 1100, and when I start the application, I provide the following switches:

 -Dcom.sun.management.jmxremote -Djava.rmi.server.hostname=w.x.y.z -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

As shown above, I set -Djava.rmi.server.hostname=w.x.y.z in order to make it possible to connect to the application through the NAT. I have also opened up the relevant ports on the machine's firewall, and set up a port forward from w.x.y.z:1100 to a.b.c.d:1100

When I then try to connect to the app from outside the network using jconsole on w.x.y.z:1100, I get a java.io.IOException: jmxrmi

 java.io.IOException: jmxrmi
at sun.tools.jconsole.ProxyClient.checkSslConfig(ProxyClient.java:236)
at sun.tools.jconsole.ProxyClient.<init>(ProxyClient.java:127)
at sun.tools.jconsole.ProxyClient.getProxyClient(ProxyClient.java:483)
at sun.tools.jconsole.JConsole$3.run(JConsole.java:524)
 Caused by: java.rmi.NotBoundException: jmxrmi
at sun.rmi.registry.RegistryImpl.lookup(RegistryImpl.java:136)
at sun.rmi.registry.RegistryImpl_Skel.dispatch(Unknown Source)
at sun.rmi.server.UnicastServerRef.oldDispatch(UnicastServerRef.java:409)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:267)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at sun.rmi.transport.Transport$1.run(Transport.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:275)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:252)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:378)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at sun.tools.jconsole.ProxyClient.checkSslConfig(ProxyClient.java:234)

If I try to connect using jconsole on w.x.y.z:1099, I get a java.rmi.ConnectException (Connection refused). How can I expose my JMX MBeans outside my NATted firewall?

 java.rmi.ConnectException: Connection refused to host: w.x.y.z; nested exception is: 
java.net.ConnectException: Operation timed out
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2373)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:297)
at sun.tools.jconsole.ProxyClient.tryConnect(ProxyClient.java:355)
at sun.tools.jconsole.ProxyClient.connect(ProxyClient.java:313)
at sun.tools.jconsole.VMPanel$2.run(VMPanel.java:292)
Caused by: java.net.ConnectException: Operation timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.<init>(Socket.java:425)
at java.net.Socket.<init>(Socket.java:208)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:147)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 9 more

Interestingly enough, I can connect to both ports (1099 and 1100) on w.x.y.z using telnet.

The relevant portion of my Spring configuration is:

<bean id="registry" class="org.springframework.remoting.rmi.RmiRegistryFactoryBean">
    <property name="port" value="1100" />
</bean>
<bean id="serverConnector" class="org.springframework.jmx.support.ConnectorServerFactoryBean" depends-on="registry">
    <property name="server" ref="mbeanServer" />
    <property name="objectName" value="connector:name=rmi" />
    <property name="serviceUrl" value="service:jmx:rmi://a.b.c.d:1100/jndi/rmi://a.b.c.d:1100/server" />
    <property name="environment">
        <props>
            <prop key="jmx.remote.jndi.rebind">true</prop>
        </props>
    </property>
</bean>
<bean id="mbeanServer" class="org.springframework.jmx.support.MBeanServerFactoryBean">
    <!-- indicate to first look for a server -->
    <property name="locateExistingServerIfPossible" value="true"/>
</bean>

Note: I am not using any servlet containers.

EDIT: The first answer provided me with the basis for the solution to my specific problem, which can be seen in the second answer.

like image 387
Siddhu Avatar asked Mar 10 '14 16:03

Siddhu


People also ask

Does JMX use RMI?

Setting this property registered the Java VM platform's MBeans and published the Remote Method Invocation (RMI) connector via a private interface to allow JMX client applications to monitor a local Java platform, that is, a Java VM running on the same machine as the JMX client.

How do I enable JMX in spring boot?

JMX is automatically enabled by default in a Spring Boot application. As a result, all of the Actuator endpoints are exposed as MBeans. And it sets us up nicely to expose any other bean in the Spring application context as an MBean.

Where is my RMI port?

The RMI services use certain default ports. The default ports are left open on the firewall on cluster nodes and remote collector nodes. Note: Ports 3091 to 3094 ports are opened in firewall by vRealize Operations for Horizon.


2 Answers

The initial port you define with com.sun.management.jmxremote.port is called a registry port and is only used to start negotiation and determine next port(s) to use for "real" communication. Java RMI mechanism uses dynamically allocated ports and in general is not compatible with firewalls.

What port is used by Java RMI connection?

That said, for JMX it is possible to work around it

a) Use system properties to lock both ports (requires Java 7)

com.sun.management.jmxremote.port
com.sun.management.jmxremote.rmi.port

b) Use custom code to request a specific port. See JConsole over ssh local port forwarding

See also:

  • Why Java opens 3 ports when JMX is configured?
  • http://realjenius.com/2012/11/21/java7-jmx-tunneling-freedom/
like image 126
anttix Avatar answered Oct 04 '22 00:10

anttix


Elaborating on the solution I identified using the links and pointers anttix gave me in his answer.

So, as mentioned in the answer above, using the system properties com.sun.management.jmxremote.port and com.sun.management.jmxremote.rmi.port is the basis for the solution.

I changed my Spring config to remove the registry and serverConnector beans, as this was not allowing me to register port 1099 (PermissionExceptions resulted in my being unable to bind to port 1099), and used the platform mbean server.

The Spring config I used was just this:

 <bean id="mbeanServer" class="java.lang.management.ManagementFactory" factory-method="getPlatformMBeanServer"/>

Then, I started the application using the following switches:

 -Dcom.sun.management.jmxremote -Djava.rmi.server.hostname=w.x.y.z -Dcom.sun.management.jmxremote.port=1099 -Dcom.sun.management.jmxremote.rmi.port=1099 -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Setting the java.rmi.server.hostname to the NAT VIP w.x.y.z is critical as we need to be able to execute remote operations on locally created remote objects from outwith the network.

Note: This solution will only work on Java 7u4 or higher.

like image 22
Siddhu Avatar answered Oct 03 '22 22:10

Siddhu