How to write a Java daemon

Question

This will be a network application that will always (or near as always as I can manage) be listening on a given port.

I'm fairly new to Java, and very new to non-web server side programming, so I'd like to get feedback from the community on my assumptions and preliminary plans.

I've read about jsvc ( http://commons.apache.org/daemon/jsvc.html ) and am currently operating on the assumption that this is the "best" way to write a daemon in java for a linux box (likely running centOS).

Can nagios be configured to monitor whether or not my daemon is running, and to alert me or the sys admin when it isn't? (I assume yes, but I'm not a very talented sys admin type)

This will be an SMPP client app (or ESME app I guess) which is why I've chosen Java as it seems to be a very mature platform for SMPP. However, I know that it's more "traditional" to write a daemon in C/C++. With modern Java, performing fairly uncomplicated tasks, am I likely to run into any major disadvantages?

What's the best way to manage deployment of new builds? Just stop the daemon and replace the binary as quickly as possible and restart?

Any other input would be greatly appreciated.

Answer 1

How to write a Java daemon that has 24/7 uptime...

We run a number of 24/365 applications on our Linux servers that just call the Java like the following -- no need for any C wrappers:

nohup java -D... -X... -jar something.jar ... < /dev/null > output.log 2>&1 &

That will put the jar running in the background (nohup ... &) with no input (< /dev/null) and the output (stdout and stderr) redirected to a logfile (> output.log 2>&1). We have distributed logging infrastructure but some console output (such as thread dumps) is still expected. These applications can run for months until we upgrade them.

Can nagios be configured to monitor whether or not my daemon is running, and to alert me or the sys admin when it isn't?

In terms of monitoring, there is much you can do. Nagios looks to have a JMX plugin for testing the information that jconsole displays. There are also a lot of native JMX logging and monitoring utilities out there. We have internal green/yellow/red indicators that can be pulled up using JMX and easily checked. I also have exported a simple JMX/HTTP service from each application to provide status information making it easy for 3rd party monitoring tools to detect failures.

This will be an SMPP client app (or ESME app I guess) which is why I've chosen Java as it seems to be a very mature platform for SMPP.

I assume you mean SMPP? If so then I see no reason why Java couldn't do a good job. Our applications do a wide variety of HTTP, UDP, SMTP, JDBC, LDAP, and other protocols in real time. We use Jgroups at lot which accomplishes a complete authenticated, encrypted, network stack in Java.

What's the best way to manage deployment of new builds? Just stop the daemon and replace the binary as quickly as possible and restart?

In terms of replacing a running binary on the fly, that it more complicated. We have VIPs up front and replace the binaries at our leisure. Our internal protocols are designed to failover. If you do not have a VIP then one thing to consider would be an orderly handoff. You boot the new jar and it talks to the application running the old jar when it is ready to bind to the new port. Then the old application unbinds and the new one binds immediately afterwards. Something like that.

Hope this helps.

Answer 2

If you really want to have something running non-stop on *nix, I recommend you have a look at daemontools.

There are some examples on how to do this here and here.

Basically svscan will spawn a process that monitors your java process from init, and every time it crashes, it gets restarted.