Using XMLDecoder to cast Encoded XML to List<T>

Question

I am writing an application that reads in a large number of basic user details in the following format; once read in it then allows the user to search for a user's details using their email:

NAME             ROLE          EMAIL
---------------------------------------------------
Joe Bloggs       Manager       [email protected]
John Smith       Consultant    [email protected]
Alan Wright      Tester        [email protected]
...

The problem I am suffering is that I need to store a large number of details of all people that have worked at the company. The file containing these details will be written on a yearly basis simply for reporting purposes, but the program will need to be able to access these details quickly.

The way I aim to access these files is to have a program that asks the user for the name of the unique email of the member of staff and for the program to then return the name and the role from that line of the file. I've played around with text files, but am struggling with how I would handle multiple columns of data when it comes to searching this large file.

What is the best format to store such data in? A text file? XML? The size doesn't bother me, but I'd like to be able to search it as quickly as possible. The file will need to contain a lot of entries, probably over the 10K mark over time.

---

EDIT: I've decided to go with the XML serialisation method. I've managed to get the code for Encoding working perfectly, but the Decoding code below does not work.

XMLDecoder d = new XMLDecoder(
               new BufferedInputStream(new FileInputStream("data.xml")));
List<Employee> list = (List<Employee>) d.readObject();
d.close();
for(Employee x : list) {
    if(x.getEmail().equals(userInput)) {
        // do stuff
    }
}

When the program hits List<Employee> list = (List<Employee>) d.readObject(); an exception is thrown claiming that "Employee cannot be cast to java.util.List". I've added a bounty to this and anyone that can help me solve this problem once and for all will get lots of lovely points.

EDIT 2: I've looked a bit more into the problem and have come across Serialization as a potential answer. If anyone can look into this for me as I've no experience with Serialization or Deserialization I'd be very grateful. It can provide an Object with no problems whatsoever, but I really need to return it in the same format as it went in (List).

EDIT 3: Ugh, this problem is really starting to drive me crazy and to be honest I'm starting to think that it's an unsolvable problem. If possible, could someone take a look at the code and help provide a solution for me?

Answer 1

Since I guess others will answer this question by advicing you to use an external database, I won't:

I suggest creating a Java Bean, i.e.

public class Employee {

    public String name;
    public String role;
    public String email;

    public Employee() {}

    public Employee(String name, String role, String email) {
        setName(name);
        setRole(role);
        setEmail(email);
    }

    public void setName(String name) {
        this.name = name;
    }
    public String getName() {
        return this.name;
    }

    // etc. for other fields

}

And use the java.beans.XMLDecoder and java.beans.XMLEncoder to serialize/deserialize an ArrayList<Employee>. (You can read more about them here: http://java.sun.com/j2se/1.4.2/docs/api/java/beans/XMLEncoder.html using an older API, because I don't know which version you use.)

You can then search this array using a foreach:

XMLDecoder d = new XMLDecoder(
               new BufferedInputStream(new FileInputStream("data.xml")));
List<Employee> list = (List<Employee>) d.readObject();
d.close();
for(Employee x : list) {
    if(x.getEmail().equals(userInput)) {
        // do stuff
    }
}

The advantage of using XML serialization over "binary" serialization, is that you can also add new fields to the Employee later, if you also provide default values for them. This makes the data flexible for future use.

More info: http://java.sun.com/products/jfc/tsc/articles/persistence4/

Update:

XMLEncoder/XMLDecoder is a better solution than a binary-serialization. I advice you to do the following.

Create a new wrapper class:

public class EmployeeList {

    private final ArrayList<Employee> list = new ArrayList<Employee>();

    public List<Employee> getList() {
        return this.list;
    }
    public setList(final List<Employee> list) {
        this.list.clear();
        this.list.addAll(list); // shallow copy
    }

    // add your search methods here, for example:
    public Employee getEmployee(String email) {
        ....
    }

}

Now you can use this EmployeeList as a wrapper. Using the following code, you perhaps can see what's wrong with the XMLDecoder, when it throws a casting exception.

XMLDecoder d = new XMLDecoder(
           new BufferedInputStream(new FileInputStream("data.xml")));
final Object o = d.readObject();
System.out.println(o.getClass());
if(o instanceof EmployeeList) {
    EmployeeList el = (EmployeeList) o;

    el.getEmployee(userInput); // TODO
}else{
    System.out.println("Wrong format.");
}

You'd have to serialize your EmployeeList too:

EmployeeList el = ...;
XMLEncoder e = new XMLEncoder(...);
e.writeObject(el);

Answer 2

How about a database? You can use either Derby or Hypersonic. You can create an embedded instance of them just for your own application use. I've used them in lots of applications where I have to manipulate large quantity of data. Hypersonic is very nice and fast. Derby is bundle with the JDK so its convenient database to use.

See this for Derby and this for Hypersonic.

Answer 3

Many approaches would work. If I wasn't going to use a database, I would store the data in a gzipped, tab-delimited file. To read the file I would use:

 BufferedReader sourceReader = new BufferedReader(new InputStreamReader(
     new GZIPInputStream(new FileInputStream(srcFile))), 4096);

 String line = null;
 while (null != (line = sourceReader.readLine()) {
     String [] colData = line.split("\t");  // alternately use java.util.Scanner 
     // Create maps for columns you want to search on.
 }
 // report results by querying map

To write to the file get a buffered writer like so:

   BufferedWriter destinationWriter = new BufferedWriter(new OutputStreamWriter(
       new GZIPOutputStream(new FileOutputStream(destination))));

   // do stuff
   destinationWriter.flush();
   destinationWriter.close();

Hope that helps....