Can't load 'mnist-original' dataset using sklearn

Question

This question is similar to what asked here and here. Unfortunately, in my case the suggested solution didn't fix the problem.

I need to work with the MNIST dataset but I can't fetch it, even if I specify the address of the scikit_learn_data/mldata/ folder (see below). How can I fix this?

In case it might help, I'm using Anaconda.

Code:

from sklearn.datasets.mldata import fetch_mldata

dataset = fetch_mldata('mnist-original', data_home='/Users/michelangelo/scikit_learn_data/mldata/')
mnist = fetch_mldata('MNIST original')

Error:

---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-5-dc4d45bc928e> in <module>()
----> 1 mnist = fetch_mldata('MNIST original')

/Users/michelangelo/anaconda2/lib/python2.7/site-packages/sklearn/datasets/mldata.pyc in fetch_mldata(dataname, target_name, data_name, transpose_data, data_home)
    168     # load dataset matlab file
    169     with open(filename, 'rb') as matlab_file:
--> 170         matlab_dict = io.loadmat(matlab_file, struct_as_record=True)
    171 
    172     # -- extract data from matlab_dict

/Users/michelangelo/anaconda2/lib/python2.7/site-packages/scipy/io/matlab/mio.pyc in loadmat(file_name, mdict, appendmat, **kwargs)
    134     variable_names = kwargs.pop('variable_names', None)
    135     MR = mat_reader_factory(file_name, appendmat, **kwargs)
--> 136     matfile_dict = MR.get_variables(variable_names)
    137     if mdict is not None:
    138         mdict.update(matfile_dict)

/Users/michelangelo/anaconda2/lib/python2.7/site-packages/scipy/io/matlab/mio5.pyc in get_variables(self, variable_names)
    290                 continue
    291             try:
--> 292                 res = self.read_var_array(hdr, process)
    293             except MatReadError as err:
    294                 warnings.warn(

/Users/michelangelo/anaconda2/lib/python2.7/site-packages/scipy/io/matlab/mio5.pyc in read_var_array(self, header, process)
    250            `process`.
    251         '''
--> 252         return self._matrix_reader.array_from_header(header, process)
    253 
    254     def get_variables(self, variable_names=None):

mio5_utils.pyx in scipy.io.matlab.mio5_utils.VarReader5.array_from_header()

mio5_utils.pyx in scipy.io.matlab.mio5_utils.VarReader5.array_from_header()

mio5_utils.pyx in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex()

mio5_utils.pyx in scipy.io.matlab.mio5_utils.VarReader5.read_numeric()

mio5_utils.pyx in scipy.io.matlab.mio5_utils.VarReader5.read_element()

streams.pyx in scipy.io.matlab.streams.FileStream.read_string()

IOError: could not read bytes

Answer 1

Unfortunately fetch_mldata() has been replaced in the latest version of sklearn as fetch_openml().

So, instead of using:

from sklearn.datasets import fetch_mldata
mnist = fetch_mldata('MNIST original')

You must use:

from sklearn.datasets import fetch_openml
mnist = fetch_openml('mnist_784')
x = mnist.data
y = mnist.target

shape of x will be = (70000,784)
shape of y will be = (70000,)

Answer 2

A quick update for the question here:

mldata.org seems to still be down. Then scikit-learn will remove fetch_mldata.

Solution for the moment: Since using the lines above will create a empty folder a the place of data_home, find the copy of the data here: https://github.com/amplab/datascience-sp14/blob/master/lab7/mldata/mnist-original.mat and download it. Then place it the ~/sklearn_data/mldata/ which is empty.

It worked for me.