Design pattern / data loading solution

Question

I have been working on a few projects which involve loading data, sometimes remotely, sometimes local, sometimes JSON, sometimes XML. The problem I am having is that due to the speed of development and the changing minds of various clients I am finding my designs are too rigid and I would like them to be more flexable. I have been trying to think of a reusable solution to data loading, and would like some advice as I imagine many of you out there have had the same problem.

What I would like to do is to create a generic LoadingOperation abstract class, which has member variables of type Parser and Loader, which have parse() and loadData() methods respectivly. The Parser and Loader classed are interfaces and classes that implement these could be XMLParser and JSONParser, LocalLoader and RemoteLoader etc. With something like this i would like to have a new class which extends LoadingOperation for each thing to be loaded, weather thats a local XML file, or remote JSON, or whatever.

The problem is is that specific Parser implementation cannot return custom data types without breaking the polymorphic behavior of the LoadingOperation class. I have been messing around with generics and declaring subclasses of LoadingOperation like

class SpecificLoader extends LoadingOperation<CustomDataType>

and doing similar things with Parser classes, but this seems a bit weird.

Does anyone have any suggestions on what im doing wrong / could be doing better. I want to be able to react quickly to changing specifications (ignoring the fact that they shouldnt be changing that much!) and have a logical seperation of code etc...

thanks for any help!

edit:question also asked here link text

Answer 1

It sounds to me that you really want something that is softly typed with as little code as possible as demands are changing too quickly. Here is how I am doing a project of mine where I use PHP for the back end. I use sql and json to get data quick out of the database and returned to the client.

Typically I do a select in the database, for each result row, I convert the row into a map where each column becomes the key with the value as the result for that column. This list of maps are then transformed into json through a generic json rutine and the json is sent from the server.

A setup like this is extremely easy to transport some data from the server to the client, but:

• You loose type safety that you could get by using hibernate/xml/remoting.
• Your clients are tightly coupled to your database schema.
• It is though wicked fast to grab data and transport it.
• If you change the query to get more data, no recompile of clients are needed unless they need to use the new data.

To give you an idea of how this looks like in PHP I do:

In my data access object:

function getAllPortal() {
    $sql = "select firstname, lastname, U.* from person, portal_user U where id=person order by firstname, lastname";
    $prep = $this->db->prepare($sql);
    return $prep->execute();
}

And in my http service (rest based) code:

    $accPerson = new AccountPerson($db);
    echo json_encode($accPerson->getAllPortal());

To do this in Java you probably need to make some framework for getting the data out in a list of maps (or some other easy structure you want to transport to the clients). I made one in PHP that does this that also allows for use of prepared statements.

Some other considerations that you could consider (even if you don't do as above) could be:

Avoid layers. Have as few of them as possible. If you use Hibernate - embrace it. Use the objects directly in your queries and convert them to json and ship it out. The layers make your code robust if several people (or clients) need to use your data - they make your code reluctant to change fast. Knowing when to do layers and when to not is the trick and resist it as long as you can. Writing layers take time.

Go for XML or even better JSON as your transport. Don't go for a schema that resist change like xml and or serialization to pojos. Pojos is great for business logic, but sucks for data transport. If your client is thin enough, don't bother deserialize your json into objects again. Use the JSON directly. Again, as with layers, the trick is to know when recreating the pojos gives business value, and when it does not. As with layers, resist to putting in that work until you see value. Writing/maintaining deserializers take time.