Importing Google Calendar data (via API v3) to Google App Engine with Java

Question

I am in the process of writing a Java REST API for a reservation system where the event data comes Google Calendar (read-only). I'm currently trying to figure out the best way to obtain and store the event data from Google Calendar in Google App Engine's JPA datastore. I also have a few requirements:

• I need to save the previous calendar data. Simply dropping everything in the database and replacing it with new data will not suffice because I want to keep a historial view of the data for statistical purposes.
• I need to notify users when the event data changes, specifically for deletions. This requires me to diff the new event data (from the API) with the old event data (from the JPA datastore).

Does anyone have any general guidance and suggestions for what to do. Am I approaching the problem the correct way by attempting to duplicate the data into a datastore? Should I just make API requests every time I need to use the data? If I were do that, is there a way to kick off some mail service to notify users of event changes from directly within Google Calendar?

Answer 1

I think trying to replicate each event's information in the datastore will be suboptimal. If you want everything then you're going to end up storing a lot of data, and I suspect the majority of it will be of no use.

If you want to store data for statistical analysis, I'd suggest deciding what the minimal set of fields are that you wish to track, and just store those. A reasonable set of fields might be: event ID, name, time, attendees, location, description, and an internal version number for bookkeeping purposes. Increment the version number each time a field changes, and you save the new field information. For extra credit, you can save a hash of the fields and not save new versions when the hash of the most recently retrieved entry matches what is already saved.

Since you're no longer storing the entire event on your server, the question of whether or not to query the API directly when you need event details is an obvious "yes".

Google Calendar does provide email notifications when Calendar events are created or changed. If you're concerned with users being notified about new events appearing on their calendars, this is easily taken care of by setting the sendNotifications field (in the call to insert a new event) to true. Similar functionality exists for update and delete calls in the API (and in the Calendar UI for when a user modifies an event).

The most difficult part I see in this implementation is deciding how your application figures out when an event changes when a user makes a modification. (Changes made by your code can be immediately reflected in the data you keep in the datastore.) If all of the events appear on a single calendar, you can poll the API for a list of events, and use the updatedMin field with the time you last queried for updates. If the events are on multiple calendars, the approach is the same, but you'll need to do one list per calendar.

Note that whatever credentials you supply to your application will need to be able to read and write to whatever calendar(s) you are modifying. As an initial approach, I'd suggest you create a master calendar for your application that is listed as an attendee on all of the events you are tracking. Events you create on behalf of users will list the users as attendees. Events users create and wish to associate with your application will need to add the master calendar as an attendee. This last point may require administrative users get some training in the correct way to interact with the system.

The alternative to the above is to have all users delegate access to their calendar to your application, but you will have to manage multiple sets of credentials and possibly weed out any events on users' calendars that you are not interested in tracking.