Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Debugging Numpy VisibleDeprecationWarning (ndarray from ragged nested sequences)

Since NumPy version 19.0, one must specify dtype=object when creating an array from "ragged" sequences. I'm faced with a large number of array calls from my own code and Pandas using threading, line-by-line debugging led me nowhere.

I'd like to figure out which call resulted in VisibleDeprecationWarning in my own code or a call from Pandas. How would I be able to debug this? I've been looking through the source and I cannot see this warning getting called in Python (only in numpy.core._multiarray_umath.cp38-win_amd64.pyd).

like image 945
misantroop Avatar asked Jul 26 '20 08:07

misantroop


People also ask

Does NumPy support ragged arrays?

I understand that numpy doesn't support ragged arrays, but it is fine for me if the resulting array simply pads each row with a specified value to make all rows the same length. To make this concrete, suppose I have set A with 3 items, B with 2 items, and C with four items.

What is ragged sequence?

When users create arrays with sequences-of-sequences, they sometimes err in matching the lengths of the nested sequences, commonly called “ragged arrays”. Here we will refer to them as ragged nested sequences.

How do I fix NumPy Ndarray object has no attribute append?

This error occurs when you attempt to append one or more values to the end of a NumPy array by using the append() function in regular Python. Since NumPy doesn't have an append attribute, an error is thrown. To fix this, you must use np. append() instead.

Is Ndarray same as NumPy array?

numpy. array is just a convenience function to create an ndarray ; it is not a class itself. You can also create an array using numpy. ndarray , but it is not the recommended way.


1 Answers

With a function that creates a ragged array:

In [60]: def foo():      ...:     print('one')      ...:     x = np.array([[1],[1,2]])      ...:     return x      ...:                                                                                              In [61]: foo()                                                                                        one /usr/local/bin/ipython3:3: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray   # -*- coding: utf-8 -*- Out[61]: array([list([1]), list([1, 2])], dtype=object) 

I get the warning, but also the expected result.

I can control the warnings.

For example to turn if off:

In [68]: np.warnings.filterwarnings('ignore', category=np.VisibleDeprecationWarning)                  In [69]: foo()                                                                                        one Out[69]: array([list([1]), list([1, 2])], dtype=object) 

Or to raise an error:

In [70]: np.warnings.filterwarnings('error', category=np.VisibleDeprecationWarning)                   In [71]: foo()                                                                                        one --------------------------------------------------------------------------- VisibleDeprecationWarning                 Traceback (most recent call last) <ipython-input-71-c19b6d9633cf> in <module> ----> 1 foo()  <ipython-input-60-6ad21d9e07b4> in foo()       1 def foo():       2     print('one') ----> 3     x = np.array([[1],[1,2]])       4     return x       5   VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray 

The error gives a traceback telling me where the warning was raised.

There may be ways of refining the warning filter to catch just this one, and not others of the same category. I haven't used this mechanism much.

Read np.warnings.filterwarnings docs for more details.

like image 70
hpaulj Avatar answered Sep 29 '22 23:09

hpaulj