Data throttling C#

Question

I am using thirdparty component which accepts maximum 25KB data at a time. I am passing array of the objects to this third party component from my application.

However the amount of data that my application writes is much more than 25KB. I am retrieving data from database and calling the component directly.

I have added reference of the component in the application. The data I am passing to the component as array of object which contains primitive and non primitive types.

How can I implement the data throttling here?

Answer 1

You can calculate the size of the one row in your database. After that every time you pass something you add that size to a variable. At the same time you are using a Stopwatch that runs. Just check if the Stopwatch.EllapsedSeconds is bigger than 1 second. If yes reset the Stopwatch and reset your variable with the size you ve already passed. If not check if the size you ve already passed (the amount of variable) is bigger than 25KB. If that is true call System.Thread.Thread.Sleep(Math.Max(1000 - StopWatch.EllapsedMilliseconds, 0)). But remember you have to do that in an extra thread because sleep blocks the whole thread.

Answer 2

You must implement a data buffer between your app and the component. The best way to do that is to:

• make a class that has it's own internal thread,
• on the public part of the class' interface implement Write method which accepts a byte array and stores it to a queue
• internal thread writes up to 25kb chunk of data from queue the and sleeps for 1 second, minus the time it took to write the chunk.

You also must take care, if the data stream is continuous and is produced at a rate > 25kbps, then you produce more then the component can consume and your queue will overflow.