Hunt down java.net.SocketException: No buffer space available

Question

Hi I have very ugly problem with: java.net.SocketException: No buffer space available (maximum connections reached?) It is client-server app. Client is Windows XP SP2 32b, with two net cards core duo. Java 1.6. u7. Application have couple server socket open for local communication and couple of client socket for rmi to jboss server.

After couple of hours/days! i am unable to open any new client socket to do communication to server. Server sockets still works.

Windows netstat shows something from 130 to 150 connection. When manually trying I exhausted buffer after ~3500 connections!

I tried:

check every socket we use that we also close it.

run netstat at background to monitor opened connections

run virus scan to found any malware

update java to 1.6 u16

disable second network interface

Once java is restarted I am able to open new connection.

Whole Exception:

cause:javax.naming.CommunicationException: Failed to connect to server IP:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server IP:1099 [Roo
t exception is java.net.SocketException: No buffer space available (maximum connections reached?): JVM_Bind]]
2009-08-03 09:13:18,968 DEBUG [Thread-9] - stack trace:
2009-08-03 09:13:18,968 DEBUG [Thread-9] - org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1562)
2009-08-03 09:13:18,968 DEBUG [Thread-9] - org.jnp.interfaces.NamingContext.lookup(NamingContext.java:634)
2009-08-03 09:13:18,968 DEBUG [Thread-9] - org.jnp.interfaces.NamingContext.lookup(NamingContext.java:627)
2009-08-03 09:13:18,968 DEBUG [Thread-9] - javax.naming.InitialContext.lookup(Unknown Source)

--edited
We finally hit the problem with malfunctioning snmp server. I wrote my notes in comment bellow. Thanks for help.

Answer 1

It certainly sounds like you are leaking Sockets somehow in your app.

• Check that your code always closes the Sockets it opens ... even in the event of some exception; i.e. do the close in a finally block.
• If your code uses URL connections, make sure that they get disconnected.
• I'm not an expert, but should your code close its InitialContext object?

Answer 2

What we tried (and successfully) kill the problem. JAVA - check again every socket we used, register them in some special class if needed
- provide SocketFactory and ServerSocketFactory for every class which open socket itself (for example jboss Connectors)
- check opened files, close them in finally
- URL opens connection too, but if you ask for stream after that, connection is closed together with stream (thanks Stephen).

OS
- use different java (1.5, 1.6, 1.7)
- install new drivers
- use netstat and monitor traffic on background (using scripts, yes win xp can do the scripts pretty nicely). Use advanced packet sniffers (wire shark?) if needed.
- win xp have limit for concurrent connections, check them (google) too
- check again and again for virus and mallware (even on private network!)