Combining java spring/thread and database access for time-critical web applications

Question

I'm developing an MVC spring web app, and I would like to store the actions of my users (what they click on, etc.) in a database for offline analysis. Let's say an action is a tuple (long userId, long actionId, Date timestamp). I'm not specifically interested in the actions of my users, but I take this as an example.

I expect a lot of actions by a lot of (different) users par minutes (seconds). Hence the processing time is crucial.

In my current implementation, I've defined a datasource with a connection pool to store the actions in a database. I call a service from the request method of a controller, and this service calls a DAO which saves the action into the database.

This implementation is not efficient because it waits that the call from the controller and all the way down to the database is done to return the response to the user. Therefore I was thinking of wrapping this "action saving" into a thread, so that the response to the user is faster. The thread does not need to be finished to get the reponse.

I've no experience in these massive, concurrent and time-critical applications. So any feedback/comments would be very helpful.

Now my questions are:

• How would you design such system? would you implement a service and then wrap it into a thread called at every action?

• What should I use? I checked spring Batch, and this JobLauncher, but I'm not sure if it is the right thing for me.

• What happen when there are concurrent accesses at the controller, the service, the DAO and the datasource level?

In more general terms, what are the best practices for designing such applications?

Thank you for your help!

Answer 1

Take a singleton object @ apps level and update it with every user action. This singleton object should have a Hashmap as generic, which should get refreshed periodically say after it reached a threshhold level of 10000 counts and save it to DB, as a spring batch.

Also, periodically, refresh it / clean it upto the last no.# of the records everytime it processed. We can also do a re-initialization of the singleton instance , weekly/ monthly. Remember, this might lead to an issue of updating the same in case, your apps is deployed into multiple JVM. So, you need to implement the clone not supported exception in singleton.

Answer 2

Here's what I did for that :

Used aspectJ to mark all the actions of the user I wanted to collect. Then I sent this to log4j with an asynchronous dbAppender...

This lets you turn it on or off with log4j logging level.

works perfectly.