Too Little CPU Utilization in Java

Question

Hey stackoverflow community!

I'm having an issue where a highly involved algorithmic program is using TOO LITTLE cpu utilization: somewhere between 3 and 4%. It is taking very long to return results, and I believe it's just not working hard enough.

Do any of you geniuses have any ideas why this would occur - if anything I would expect 100% utilization. One additional detail is that the program makes inserts into a sqlite3 database, and thus yes, there are a lot of JNI calls via the sqlite3jdbc library I believe. (Note that I wanted to defer these inserts with a PreparedQuery batch earlier, but this caused major memory problems - there's a lot of data).

Thanks in advance

UPDATE: Fixed. Yeah, I was just being a doofus, but I didn't expect that sqlite would start a new transaction and do so much overhead.

I now use a PreparedStatement and queue 32768 entries before insert - seemed like a good enough number to me.

Answer 1

I would never recommend that someone use a JDBC driver with JNI if a type IV, 100% Java version is available. Google found this one.

With that aside, I can't tell anything without more info. Are the app and the database running on the same hardware?

What is so "intensive" about INSERTs?

I'd recommend profiling and getting some real data rather than guessing. Faith-based computing never works for me.

Answer 2

Obviously the database calls are causing delays. Isn't it an option to create smaller batches and test if that helps?? Maybe you could parallelize the algorithm as well to have a queue somewhere taking results and another thread cleaning out that queue?

edit:

There are also some other problem areas:

• Database optimalization (model)
• Database server configuration
• Disk speed

All these factors should be taken into account

Answer 3

If you're writing a lot of data, then it sounds like you may be disk bound. Take a look at your disk io stats on the machine, and if that's actually the bottleneck, either find hardware with better io, or figure out how to do less writes.