Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MemoryError with numpy arange

I want to create an array of powers of 10 as a label for the y axis of a plot. I am using the plt.yticks() with matplotlib imported as plt but this does not matter here anyway. I have plots where as the y axis is varying from 1e3 to 1e15. Those are log plots. Matplotlib is automatically displaying those with ticks with 1e2 steps and I want to have a step of 10 instead (in order to be able to use the minorticks properly).

I want to use the plt.yticks(numpy.arange(1e3, 1e15, 10)) command as said but numpy.arange(1e3, 1e15, 10) result in a MemoryError. Isn't it supposed to output an array of length 13? Why does the memory gets full?

How to overpass this issue and not build the array manually?

I also tried using built-in range but it won't work with floats.

Thank you.

like image 249
Odyseus_v4 Avatar asked Sep 18 '18 09:09

Odyseus_v4


People also ask

What does the NumPy function arange do?

Return evenly spaced values within a given interval. arange can be called with a varying number of positional arguments: arange(stop) : Values are generated within the half-open interval [0, stop) (in other words, the interval including start but excluding stop).

How does NumPy store data?

NumPy arrays consist of two major components: the raw array data (from now on, referred to as the data buffer), and the information about the raw array data. The data buffer is typically what people think of as arrays in C or Fortran, a contiguous (and fixed) block of memory containing fixed-sized data items.

How do you get memory consumed by each item in the array in NumPy?

Using nbytes attribute of NumPy array. nbytes: This attribute gives the total bytes consumed by the elements of the NumPy array.


3 Answers

Try the logspace from NumPy as

plt.yticks(numpy.logspace(3, 15, 13))

Here you give the starting and the last exponent (powers of 10) and the number of data-points in between. If you print the above mesh, you get the following

array([1.e+03, 1.e+04, 1.e+05, 1.e+06, 1.e+07, 1.e+08, 1.e+09, 1.e+10,
   1.e+11, 1.e+12, 1.e+13, 1.e+14, 1.e+15])
like image 97
Sheldore Avatar answered Oct 18 '22 06:10

Sheldore


You could also just do:

10. ** np.arange(3,16)

that decimal point is important, as without it you will overflow the default int32 dtype for integers

like image 31
Daniel F Avatar answered Oct 18 '22 06:10

Daniel F


An alternative way to do this, rather than explicitly defining the tick positions, is to use a LogLocator from the matplotlib.ticker module, and manually increase the number of ticks (by default, it will try to set a nice-looking number of ticks; i.e. so it doesn't look too cramped).

In this example, I set the number of ticks to 13 on the Axes on the right (using numticks=13), and you can see this increases the number of ticks so there is one on each integer power of 10.

import matplotlib.pyplot as plt
import matplotlib.ticker as ticker

# Create figure and axes
fig, (ax1, ax2) = plt.subplots(ncols=2)

# Make yscale logarithmic
ax1.set_yscale('log')
ax2.set_yscale('log')

# Set y limits
ax1.set_ylim(1e3, 1e15)
ax2.set_ylim(1e3, 1e15)

# On ax2, lets tell the locator how many ticks we want
ax2.yaxis.set_major_locator(ticker.LogLocator(numticks=13))

ax1.set_title('default ticks')
ax2.set_title('LogLocator with numticks=13')

plt.show()

enter image description here

EDIT:

To add minor ticks with this method, we can use another LogLocator, and this time set the subs option to say where we want minor ticks in each decade. Here I haven't set minor ticks on every 0.1 because it would be too cramped, so just done for a subset. Note that if you set minor ticks like this, you also need to turn off tick labels for the minor ticks, using a NullFormatter.

ax2.yaxis.set_major_locator(ticker.LogLocator(numticks=13))
ax2.yaxis.set_minor_locator(ticker.LogLocator(subs=(0.2,0.4,0.6,0.8),numticks=13))
ax2.yaxis.set_minor_formatter(ticker.NullFormatter())

enter image description here

like image 1
tmdavison Avatar answered Oct 18 '22 08:10

tmdavison