Parsing framework that deals well with circular references in JSON [closed]

Question

I am working on an Android project and am currently trying to figure out how to deserialize some JSON from our APIs that includes reference cycles into an object graph, which I can then manipulate and store in a database. Let me give an example:

{
    "id": "24",
    "name": "Bob",
    "friends": [
        {
            "id": "13",
            "name": "Alice",
            "friends": [
                {
               "id": "24" // and we have a circular reference
                }
            ]
        }
    ]
}

Here, a person object called Bob is friends with person Alice, and Alice is in turn friends with Bob. Since the relationship is recursive, Alice’s friends relationship to Bob is not realized as a full person object anymore but only his id is provided.

What tools do you use to perform the above mentioned steps? I tried to implement the object mapping part with Jackson but failed to find a solution for the cycle requirement. I found an ongoing discussion about this topic that mentions JSOG which might be helpful, but our APIs are fixed and not JSOG compliant.

Basically what I am looking for is something like RestKit (iOS framework) for Android.

Answer 1

Once API is fixed, I'd implement it in this manner:

From DB perspective, I'd have 2 tables - UserTable and RelationsTable to keep all edges of your friends graph:

I.e. the idea is to keep Users in the one table and their relations in Relations table. It allows also to add some extra logic on top of it later (for example, user hides his connection or blocks someone, etc. - any possible edges of the graph). Also, it allows to mitigate issues with circular references.

As a framework to retrieve data from service & parse jsons, I'd use Retrofit.

First, I'd define UserBase and User classes:

public class UserBase {
    public string id;
}

public final class User extends UserBase {
    public string name;
    public List<UserBase> friends;
    // user's "real" friends, not just ids, fills from SQLite
    public List<User> userFriends;
}

where, as you can see, friends is a list of UserBase objects for Retrofit to parse the object from JSON and userFriends - the list, which we'll fill from SQLite manually in further steps.

Now, let's define some help-classes to operate with DBs:

public interface Dao<TItem> {
    void add(List<TItem> items);
    void removeAll();
    List<TItem> getAll();
}
....
public abstract class AbstractDao<TItem> implements Dao<TItem> {
    protected final SQLiteDatabase database;
    protected final SqlUtilities sqlUtilities;

    public AbstractDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        this.database = database;
        this.sqlUtilities = sqlUtilities;
    }
}

Now we need Dao's for RelatedTable and for UserTable:

public class UserRelation {
    public String mainUserId;
    public String relatedUserId;
}
...
public interface UserRelationDao extends Dao<UserRelation> {
    ...
    List<User> getFriends(String userId);
    ...
}
... 
public interface UserDao extends Dao<User> {
    ...
    void addWithIgnore(List<TItem> items);
    void update(List<TItem> items);
    void upsert(List<TItem> items);

    User getById(String userId);
    ...
}

Once it's done, we can actually implement this interfaces:

DefaultUserRelationDao class:

public class DefaultUserRelationDao extends AbstractDao<UserRelation> implements UserRelationDao {
    static final String MAIN_USER_COLUMN = "mainuser";
    static final String RELATED_USER_COLUMN = "relateduser";

    private static final String[] COLUMN_NAMES = new String[]{
            MAIN_USER_COLUMN,
            RELATED_USER_COLUMN,
    };

    private static final String[] COLUMN_TYPES = new String[]{
            "TEXT",
            "TEXT",
    };

    private static final String TABLE = "userrelation";
    static final String CREATE_TABLE = SqlUtilities.getCreateStatement(TABLE, COLUMN_NAMES, COLUMN_TYPES);
    static final String ALL_CONNECTED_USERS =
            "SELECT " + Joiner.on(",").join(DefaultUserDao.COLUMN_NAMES) +
                    " FROM " + UserTable.TABLE_NAME + "," + TABLE +
                    " WHERE " + RELATED_USER_COLUMN + "=" + DefaultUserDao.USER_ID_COLUMN;

    public DefaultUserRelationDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        super(database, sqlUtilities);
    }

    @Override
    public void add(List<UserRelation> userRelations) {
        try {
            database.beginTransaction();
            ContentValues contentValues = new ContentValues();

            for (UserRelation relation : userRelations) {
                sqlUtilities.setValuesForUsersRelation(contentValues, relation);
                database.insertOrThrow(TABLE, null, contentValues);
            }

            database.setTransactionSuccessful();
        } finally {
            database.endTransaction();
        }
    }

    @Override
    public List<User> getFriends(String userId) {
        Cursor cursor = database.rawQuery(ALL_CONNECTED_USERS, new String[]{userId});
        return sqlUtilities.getConnectedUsers(cursor);
    }
}

and DefaultUserDao class:

public final class DefaultUserDao extends AbstractUDao<User> implements UserDao {

    public static final String USER_ID_COLUMN = "userid";
    static final String USER_NAME_COLUMN = "username";

    public static final String[] COLUMN_NAMES = new String[]{
            USER_ID_COLUMN,
            USER_NAME_COLUMN,
    };

    private static final String TABLE = "users";
    private static final String SELECT_BY_ID =
            SqlUtilities.getSelectWhereStatement(TABLE, COLUMN_NAMES, new String[]{ USER_ID_COLUMN });

    static final String CREATE_TABLE = SqlUtilities.getCreateStatement(TABLE, COLUMN_NAMES, COLUMN_TYPES);

    public DefaultUserDao(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        super(database, sqlUtilities);
    }

    @Override
    public void add(List<User> users) {
        try {
            database.beginTransaction();
            ContentValues contentValues = new ContentValues();

            for (User user : users) {
                sqlUtilities.setValuesForUser(contentValues, user);
                database.insertOrThrow(UserTable.TABLE_NAME, null, contentValues);
            }

            database.setTransactionSuccessful();
        } finally {
            database.endTransaction();
        }
    }

    @Override
    public User getById(String userId) {
        return getUserBySingleColumn(SELECT_BY_ID, userId);
    }
    .....
    private User getUserBySingleColumn(String selectStatement, String value) {
        Cursor cursor = database.rawQuery(selectStatement, new String[]{value});
        List<User> users = sqlUtilities.getUsers(cursor);
        return (users.size() != 0) ? users.get(0) : null;
    }
}

To create our tables, we need to extend SQLiteOpenHelper and in onCreate() actually create tables:

public final class DatabaseHelper extends SQLiteOpenHelper {
    static final String DATABASE_NAME = "mysuper.db";
    public DatabaseHelper(Context context) {
        super(context, DATABASE_NAME, null, SCHEMA_VERSION);
    }

    @Override
    public void onCreate(SQLiteDatabase db) {
        db.execSQL(DefaultUserDao.CREATE_TABLE);
        db.execSQL(DefaultUserRelationDao.CREATE_TABLE);
    }
    ...
}

Now, I'd suggest to define LocalStorage interface with all possible actions with cache:

• get all users
• get user by id
• add users
• add connection between users

• etc.

  public interface LocalStorage {
     User getUserById(String userId);
     void addUsers(List<User> users);
     ....
  }
  

and it's implementation:

public final class SqlLocalStorage implements LocalStorage {

    private UserDao userDao;
    private UserRelationDao userRelationDao;

    private SQLiteDatabase database;
    private final Object initializeLock = new Object();
    private volatile boolean isInitialized = false;
    private SqlUtilities sqlUtilities;

    // there database is
    //    SQLiteOpenHelper helper = new DatabaseHelper(context);
    //    database = helper.getWritableDatabase();
    public SqlLocalStorage(SQLiteDatabase database, SqlUtilities sqlUtilities) {
        this.database = database;
        this.sqlUtilities = sqlUtilities;
    }

    @Override
    public User getUserById(String userId) {
        initialize();

        User user = userDao.getById(userId);
        if (user == null) {
            return null;
        }

        List<User> relatedUsers = userRelationDao.getFriends(userId);
        user.userFriends = relaterUsers;
        return user;
    }

    @Override
    public void addUsers(List<User> users) {
        initialize();

        for (User user : users) {
            for (UserBase friend : user) {
                UserRelation userRelation = new UserRelation();
                userRelation.mainUserId = user.id;
                userRelation.relatedUserId = friend.id;

                UserRelation userRelationMutual = new UserRelation();
                userRelationMutual.mainUserId = friend.id;
                userRelationMutual.relatedUserId = user.id;

                userRelationDao.add(userRelation);
                userRelationMutual.add(userRelation)
            }
        }

        userDao.addWithIgnore(users);
    }

    void initialize() {
        if (isInitialized) {
            return;
        }

        synchronized (initializeLock) {
            if (isInitialized) {
                return;
            }

            Log.d(LOG_TAG, "Opens database");
            userDao = new DefaultUserDao(database, sqlUtilities);
            userRelationDao = new DefaultUserRelationDao(database, sqlUtilities);
            isInitialized = true;
        }
    }
}

Last step - the actual usage of it:

//somewhere in non-UI thread
List<User> users = dataSource.getUsers();
localStorage.addUsers(users);
final User userBob = localStorage.getUserById("42");

NB! I'm heavily using here my custom class SqlUtilities. Unfortunately, it's way too big to post it here, but just an example to give some ideas what's inside - here's how getUsers(Cursor cursor) looks there:

.....
public List<User> getUsers(Cursor cursor) {
    ArrayList<User> users = new ArrayList<>();
    try {
        while (cursor.moveToNext()) {
            users.add(getUser(cursor));
        }
    } finally {
        cursor.close();
    }

    return users;
}

private User getUser(Cursor cursor) {
    User user = new User(cursor.getString(0));
    user.FullName = cursor.getString(1);
    ....
    return user; 
}
.....

I hope, you'll forgive me skipping some details (especially, regarding case, when DB has to be updated, when data is not full and besides getting it from cache, you have to retrieve it from server first, and then load it into the cache, etc). If any crucial part is missing - please, post it in comments and i'll be glad to update the post.

I hope, it will help you.

Answer 2

You can have a look into JSON-RPC. This is a good framework which supports JSON parsing and object mapping of complex object relationship.