I'm using numpy to create a cube array with sides of length 100, thus containing 1 million entries total. For each of the million entries, I am inserting a 100x100 matrix whose entries are comprised of randomly generated numbers. I am using the following code to do so:
import random
from numpy import *
cube = arange(1000000).reshape(100,100,100)
for element in cube.flat:
matrix = arange(10000).reshape(100,100)
for entry in matrix.flat:
entry = random.random()*100
element = matrix
I was expecting this to take a while, but with 10 billion random numbers being generated, I'm not sure my computer can even handle it. How much memory would such an array take up? Would RAM be a limiting factor, i.e. if my computer doesn't have enough RAM, could it fail to actually generate the array?
Also, if there is a more efficient to implement this code, I would appreciate tips :)
The size in memory of numpy arrays is easy to calculate. It's simply the number of elements times the data size, plus a small constant overhead. For example, if your cube. dtype is int64 , and it has 1,000,000 elements, it will require 1000000 * 64 / 8 = 8,000,000 bytes (8Mb).
There is no general maximum array size in numpy.
NumPy uses much less memory to store dataThe NumPy arrays takes significantly less amount of memory as compared to python lists. It also provides a mechanism of specifying the data types of the contents, which allows further optimisation of the code.
Less memory usage: The Python NumPy array consumes less memory than lists. Less execution time: The NumPy array is pretty fast in terms of execution, as compared to lists in Python.
for the "inner" part of your function, look at the numpy.random module
import numpy as np
matrix = np.random.random((100,100))*100
A couple points:
cube.dtype
is int64
, and it has 1,000,000 elements, it will require 1000000 * 64 / 8 = 8,000,000
bytes (8Mb).cube
, element = matrix
will simply overwrite the element
variable, leaving the cube
unchanged. The same goes for the entry = random.rand() * 100
.If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With