How can I retrieve the full newsfeed of a user via the Facebook API?

Question

I would like to retrieve the full newsfeed including historical data of a given user. In principal, this is straight forward using either an authenticated call to the Graph API or to the FQL API.

With the Graph API, I access the me/home endpoint. This results in 25 entries. I can iterate over the pages and retrieve around 8 pages back into history giving me around 200 entries. I write around 200 entries, because with each run through this I get a different number of total entries. Sometimes more, sometimes less.

With the FQL API, I call SELECT post_id, created_time, actor_id, message FROM stream WHERE filter_key = 'nf' AND is_hidden=0 AND created_time > 1262304000 LIMIT 500 where the created time reflects 1 Jan 2010. This gives me around 150 entries.

Both methods don't seem to allow to work your way backwards into history. In the FQL query, I also tried to play around with the created_time field and LIMIT to go backwards in small chunks but it didn't work.

The documentation of the stream table http://developers.facebook.com/docs/reference/fql/stream/ says somehow cryptically:

The profile view, unlike the homepage view, returns older data from our databases.

Homepage view - as far as I understand - is another word for Newsfeed, so that might mean that what I want is not even possible at all?

To make things worse (but that's not the main topic of this question) the returned datasets from the two methods differ. Both contain entries that the other does not show but they also have many entries in common. Even worse, the same is true in comparison to the real newsfeed on the Facebook website.

Does anyone have any experience or deeper insights on this?

Answer 1

Maybe I am mis-understanding your question, but can't you simply call the graph api with /me/home?limit=5000 and then ?limit=5000&offset=5000 or whatever the max limit value Facebook allows is?