Java reading and writing to a local database?

Question

I have a program which is constantly getting information from a website and is constantly updating. As of now I store all that information in an arraylist and then when I am done I write it to a text file.

I need to manipulate that information; however, it creates a massive text file and I can't constantly read and write information to a text file because it takes too long. So someone told me to look into using a database. The only database I have ever used was a MySQL database for a website, never with java.

Is there any way to make a database local? As in only on my computer (don't want to pay for web hosting when I am the only one accessing this information at my computer)? I was looking a little bit into SQLite but one of the things I saw was that it does not allow concurrency. I'm thinking of multi-threading my program and having it read and write different sections at the same time. Is this possible?

Basically what I'm asking for here is:

1. What type of database should I use?
2. How do I install that database on my computer?
3. Some information on how to use Java jdbc?(was reading a little into it before)
4. Any tutorials on any of the above (video, text, etc)

Answer 1

It sounds like your friend is correct and a database would greatly help you improve performance.

1. You can really use any type of database. You can even continue using MySQL by installing it locally and using Connector/J to connect in Java. Though embeddable databases are probably the easiest. One option is Hypersonic DB (HSQLDB)
2. There is no need to "install" embeddable databases. Simply add their library (.jar file) to your project's build path. When you "connect" to the database, you can usually tell the database to use a location on your filesystem to store its data.
3. JDBC stands for Java DataBase Connectivity. It is the API used by Java applications to communicate with databases. One of the important features of JDBC is that it (usually) does not matter which database you are talking to as the API for accessing any database is the same*.
4. Here is one tutorial for JDBC: JDBC(TM) Database Access

*The API for accessing databases--e.g. sending queries and retrieving results--will generally be the same but JDBC can't hide differences in things like SQL. If you use an ORM like Hibernate, it can usually make it much easier to handle all the little incompatibilities between databases.

Answer 2

You need embeddable Java database, e.g. HSQL.