Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using download_data() and untar_data() in fastai library

I downloaded Fashion MNIST dataset from kaggle using dowload_data() function in fastai library.

downloaded_data = download_data("https://www.kaggle.com/zalando-research/fashionmnist/download")

output -

PosixPath('/root/.fastai/data/download.tgz')

download_data saves it as .tgz file, now I use untar_data().

path = untar_data('/root/.fastai/data/download.tgz')

output -

PosixPath('/root/.fastai/data/download.tgz')

Which did not extract .tgz file. How do I use this dataset in fastai library?

like image 322
AshishSinha5 Avatar asked Oct 16 '22 08:10

AshishSinha5


1 Answers

In fastai library, the download_data gives you a pathlib.PosixPath file, not the exact file, you need to use another unzipping library to extract the data.

If you just need the MNIST data from fast ai, here's an easier way:

from fastai import datasets
import gzip, pickle
MNIST_URL='http://deeplearning.net/data/mnist/mnist.pkl'
path = datasets.download_data(MNIST_URL, ext='.gz')
with gzip.open(path, 'rb') as f:
    ((x_train, y_train), (x_valid, y_valid), _) = pickle.load(f, encoding='latin-1')
like image 53
user140536 Avatar answered Oct 21 '22 07:10

user140536