Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python/sklearn - preprocessing.MinMaxScaler 1d deprecation

I'd like to scale a column of a dataframe to have values between 0 and 1. For this I'm using a MinMaxScaler, which works fine, but is sending me mixed messages. I'm doing:

x = df['Activity'].values #returns a numpy array
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['Activity'] = pd.Series(x_scaled)

Message numero uno for this code is a warning:

DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.

Okay, so apparentyl having 1d arrays is gonna be a no-no soon, so let's try to reshape it as advised:

x = df['Activity'].values.reshape(-1, 1)

Now the code doesn't even run: Exception: Data must be 1-dimensional is thrown. So I'm confused. 1d is going to be deprecated soon, but the data also has to be 1d?? How to do this safely? What's the issue here?

EDIT as requested by @sascha

x looks like this:

array([ 0.00568953,  0.00634314,  0.00718003, ...,  0.01976002,
        0.00575024,  0.00183782])

And after reshaping:

array([[ 0.00568953],
       [ 0.00634314],
       [ 0.00718003],
       ..., 
       [ 0.01976002],
       [ 0.00575024],
       [ 0.00183782]])

The whole warning:

/usr/local/lib/python3.5/dist-packages/sklearn/preprocessing/data.py:321: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
  warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)
/usr/local/lib/python3.5/dist-packages/sklearn/preprocessing/data.py:356: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and will raise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample.
  warnings.warn(DEPRECATION_MSG_1D, DeprecationWarning)

The error when I reshape:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-132-df180aae2d1a> in <module>()
      2 min_max_scaler = preprocessing.MinMaxScaler()
      3 x_scaled = min_max_scaler.fit_transform(x)
----> 4 telecom['Activity'] = pd.Series(x_scaled)

/usr/local/lib/python3.5/dist-packages/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    225             else:
    226                 data = _sanitize_array(data, index, dtype, copy,
--> 227                                        raise_cast_failure=True)
    228 
    229                 data = SingleBlockManager(data, index, fastpath=True)

/usr/local/lib/python3.5/dist-packages/pandas/core/series.py in _sanitize_array(data, index, dtype, copy, raise_cast_failure)
   2918     elif subarr.ndim > 1:
   2919         if isinstance(data, np.ndarray):
-> 2920             raise Exception('Data must be 1-dimensional')
   2921         else:
   2922             subarr = _asarray_tuplesafe(data, dtype=dtype)

Exception: Data must be 1-dimensional
like image 371
lte__ Avatar asked Nov 18 '16 19:11

lte__


1 Answers

You can simply drop pd.Series:

import pandas as pd
from sklearn import preprocessing
df = pd.DataFrame({'Activity': [ 0.00568953,  0.00634314,  0.00718003, 
                                0.01976002, 0.00575024,  0.00183782]})
x = df['Activity'].values.reshape(-1, 1) #returns a numpy array
min_max_scaler = preprocessing.MinMaxScaler()
x_scaled = min_max_scaler.fit_transform(x)
df['Activity'] = x_scaled

or you can explicitly get first column of x_scaled:

df['Activity'] = pd.Series(x_scaled[:, 0])
like image 92
Ilya V. Schurov Avatar answered Sep 23 '22 00:09

Ilya V. Schurov