What collection classes to use to store long strings?

Question

I will be getting a JSON string every .01 seconds or even faster and I have to insert it in some collection so that I can later loop through every row and do some processing and delete that row after that. The process of insertion keeps on going. I am a bit confused as whether to use ArrayList or Queue.

So basically I would insert the first message then second and then ... and simultaneously another process would read the first inserted value for processing and deleting that record followed by second inserted value. Please advise which one would be fastest and takes less storage? Or any other collection would best suit my requirement?

Edit: I have two methods -

GetMessages() - which keeps on getting/adding messages in queue (here the messages are received from only one source which keeps on sending messages and does not stop till we manually stop the process)

ProcessMessages() - which will read the queue records and after processing delete them from the queue

Answer 1

Sounds like you should use a ConcurrentQueue<string>, where one thread can push values that another thread could pop.

Definitely don't use or even think of using the pre-generic ArrayList type ever again.

You'd still need some bookkeeping code that checks whether your processing is running faster than the insertion, otherwise you're guaranteed to run out of memory at some point.

Answer 2

It seems that you are looking for a Producer-Consumer design pattern (Wikipedia Producer–consumer_problem).

1. Producer creates Json's
2. Consumer processes them

If it's your case, try BlockingCollection which is specially designed for this.