I wrote a Python program that handles very large data. As it processes the data, it puts the processed data into an array, which easily grows to hundreds of megabytes or even over a gigabyte.
The reason I set it like that is because Python needs to continuously access the data in the array. Because the array gets larger and larger, the process is easily prone to error and very slow.
Is there a way to have the array-like database stored on a different file or database module and access it on a as-needed basis?
Perhaps this is a very basic task, but I have no clue.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With