json.net and incremental processing of the twitter streaming API

Question

I'm trying to make use of the Twitter streaming API in an incremental fashion to process root objects one by one and return each one via a callback asynchronously. I've seen implementations counting the number of { vs } and waiting for the numbers to balance at 0 then processing that chunk of data although to me this really just didn't feel safe.

My plan was to make use of a json.net JsonTextReader and use TokenType and Depth to do this job. This is working well in that it is a very reliable way of splitting the read at the end of each object, I just cannot figure out how to get that data into the Deserialize method. The Deserialize method does accept a JsonTextReader as an argument but that processes from that point onwards (it's a forward only stream after all) so I'm stuck. There is also a DeserializeObjectAsync method available but that only accepts a string.

var jsonSerializer = new JsonSerializer();
using (StreamReader reader = new StreamReader(_response.GetResponseStream()))
{
    using (var jsonReader = new JsonTextReader(reader))
    {
        while (jsonReader.Read())
        {
            if (jsonReader.Depth == 0 && jsonReader.TokenType == JsonToken.EndObject)
            {

                //newTweetCallback(jsonSerializer.Deserialize<Tweet>([Some magic here. Help!]));
            }
        }
    }
}

I'm hoping I'm just missing something obvious as a result of staring at the particular application for too long.

Answer 1

Actually, instead of handling the depth yourself, you can pass the partially used JsonTextReader to Deserialize, which will not read further than to the end token:

while (jsonReader.Read())
{
    if (jsonReader.TokenType == JsonToken.StartObject)
    {
        newTweetCallback(jsonSerializer.Deserialize<Tweet>(jsonReader));
        // Debug.Assert(jsonReader.TokenType == JsonToken.EndObject);
    }
}

So there's no need to read ahead yourself.