I have a large set of data that I want to cycle through in order to determine various statistics on the data set from a point in time 'D1' to a point in time in the future 'D2'. Basically, I want to add to a database each time the difference between the values is larger than 10. For example:
Datum[] data = x;
for( Datum d1 : data ){
Datum[] tail = y; //From d1 up to 10 elements ahead
for( Datum d2 : tail ){
//Calculate difference
if( (d2.val - d1.val) > 10 ){
//Insert into database
}
}
}
My question is, is there a better algorithm/method to doing this? Since 9 elements from tail are reused in the next iteration of the outer loop, can I benefit somehow from that? My goal was to get this down to much less than (Big-O Notation) O(n2), but I can't wrap my head around it. Typically finding a D1, D2 pair that satisfies the criteria means that the next D1 element will have a greater chance of matching as well. Can I use that to my advantage?
I'm trying to get this to be as efficient as possible because the data set is just so large.
An index-based for loop might perform much better than an iterator, since you can index the original array directly and avoid copying to a new array. You'd have much better memory locality, less chance of false sharing, etc.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With