Can't find kaggle.json file in google colab

Question

I'm trying to download the kaggle imagenet object localization challenge data into google colab so that I can use it to train my model. Kaggle uses an API for easy and fast access to their datasets. (https://github.com/Kaggle/kaggle-api) However, when calling the command "kaggle competitions download -c imagenet-object-localization-challenge" in google colab, it can't find the kaggle.json file which contains my username and api-key.

I haven't had this problem on my mac when running a jupyter notebook, but since I want to use google's gpu for my model, I started using google colab. Because the kaggle API expects the username and api-key to be in a kaggle.json file located in a .kaggle directory, I first created the directory .kaggle and then the file kaggle.json, into which I wrote my username and api-key (The example below doesn't display my username and api-key). I then tried to configure the path to my json file for kaggle to use when calling the kaggle download command.

!pip install kaggle  !mkdir .kaggle !touch .kaggle/kaggle.json  api_token = {"username":"username","key":"api-key"}  import json import zipfile import os with open('/content/.kaggle/kaggle.json', 'w') as file:     json.dump(api_token, file)  !chmod 600 /content/.kaggle/kaggle.json !kaggle config path -p /content 

However, when running the last command, I got the error:

IOError: Could not find kaggle.json. Make sure it's located in /root/.kaggle. Or use the environment method. 

My goal was to use the following commands to get the dataset from kaggle:

!kaggle competitions download -c imagenet-object-localization-challenge os.chdir('/content/competitions/imagenet-object-localization-challenge') for file in os.listdir():     zip_ref = zipfile.ZipFile(file, 'r')     zip_ref.extractall()     zip_ref.close() 

I don't understand why the kaggle API can't find my json file. How can I use the API in google colab?

Answer 1

According to kaggle api documentation the location where credentials json is looking for is ~/.kaggle/kaggle.json as google colab environment is Linux based. In your snippet you try to config path parameter, but it is not used to looking for credential json:

- path: Folder where file(s) will be downloaded, defaults to current working directory

So the full working snippet for google colab environment would be:

!mkdir ~/.kaggle !touch ~/.kaggle/kaggle.json  api_token = {"username":"username","key":"api-key"}  import json  with open('/root/.kaggle/kaggle.json', 'w') as file:     json.dump(api_token, file)  !chmod 600 ~/.kaggle/kaggle.json 

And then some api call like

!kaggle datasets download -d datamunge/sign-language-mnist