Which is faster?

Question

is opening a large file once reading it completely once to list faster (or) opening smaller files whose total sum of size is equal to large file and loading smaller file into list manupalating one by one faster?

which is faster?? is the difference is time large enough to impact my program?? total time difference of lesser then of 30 sec is negligible for me

Answer 1

It depends if your data fit in your available memory. If you need to resort to paging, or virtual memory, then opening a single giant file might become slower than opening more smaller files. This will be even more true if the computation you need to make creates intermediate variables that won't fit in the physical RAM either.

So, as long as the file is not that big, one opening will be faster, but if this is not true, then many opening may be faster.

At last, note that if you can do many opening, you might be able to do them in parallel and process various parts in different processes, which might make things faster again.