I'm parsing an online feed (tcp relay) that sends approximately 30-50 messages per second (300-500 rows of data). The messages contain 2 types of information: orders and history.
So, with orders, each one has a unique ID and I've got:
private static Dictionary<long,MarketOrder> MarketOrders = new Dictionary<long,MarketOrder>();
to which I insert orders as they come in. Data comes from cache files so messages can contain data which is old and has to be filtered out. I'm currently doing this:
if (MarketOrders.ContainsKey(order.OrderID))
{
// If record exists in a dictionary add hits and overwrite the object if newer.
int hits = MarketOrders[order.OrderID].Hits;
if (MarketOrders[order.OrderID].LastUpdated < order.LastUpdated)
{
MarketOrders[order.OrderID] = order;
}
MarketOrders[order.OrderID].Hits = hits + 1;
}
else
{
// If not, add new one
order.Hits = 1;
MarketOrders.Add(order.OrderID, order);
}
This runs in a BackgroundWorker process, and when the dictionary item count hits 2500, it is deep cloned (using binary serializer), cleared and another background process is initiated, that inserts the cloned copy into the database. As soon as dictionary is cleared, orders are being inserted again. So basically I'm trying to receive as much as I can and insert to database in batches.
I'm trying to do something similar with the history data. There is no unique ID as such and uniqueness comes from combination of <int, int, DateTime>
values.
I need a fast method of generating a unique key from these 3 values, so I can store it in a dictionary like I do with orders, or another method of storing and filtering that data.
Any suggestions? I'm targeting .NET 4.0.
Ticks hashcode This uses the DateTime. Now. Ticks property, which is “the number of 100-nanosecond intervals that have elapsed since 12:00:00 midnight, January 1, 0001”. It will therefore always be unique, unless the id is generated in a threaded scenario.
The key of a Dictionary
doesn't have to be a simple type. In your case, the simplest solution would be to use Tuple<int, int, DateTime>
as the key. An alternative would be create custom type which correctly implements Equals()
and GetHashCode()
(and ideally also IEquatable
).
You can do the same on the database side, most databases support compound keys.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With